* Some mprotect fixes
The biggest thing here is preventing mprotect on memory that isn't mapped in address space. This would cause exceptions before, but succeeds on real hardware.
I've also included a couple other minor fixes, mostly based around some tests I recently performed.
Note: All changes to memory pools in this PR are assumed. I have not yet tested memory pools with any of this logic, but I do at least want to prevent mprotect on pool reserved memory to avoid crashes.
* Update memory.cpp
* clang
Reserved memory counts here, so we need to use !IsFree instead of IsMapped.
I swear this is like the 10th time I've messed this sorta thing up. Seems like it's the last case of this type of mistake in our current code though.
* SearchFree adjustments
* Robust address validation
I've adjusted IsValidAddress to take in a size, and check whether the whole range is contained in vma map.
If no size is provided, the function reverts to the old form of address validation instead.
* Map around gaps
As is, this should work mostly.
Only remaining issue is adding logic to pass the "mapped regions" to the guest vma map (and make such logic cross-platform).
* Initialize vma_map using gaps
This should allow memory code to catch any issues from address space gaps, and prevent non-fixed mappings from jumping to a location that isn't actually available.
* Clang
* Fix compile
* Clang
* Fix compile again
* Set system_managed_base and system_managed_size based on
Many places in our code use system_managed_base as the minimum mappable address, ensure this fact remains the same on Windows to prevent potential bugs.
* Reduce address validation in SearchFree
Allows SearchFree to function when a certain Windows GPU driver goes and reserves the whole system managed area.
Since SearchFree is only called on flexible addresses, allowing this particular case, where addresses are in bounds, but there's not enough space to map, should be safe enough.
* Bump address space size further
To handle Madden NFL 16 (and any games like it)
* More thorough logging of available memory regions
Should help with spotting weirdness.
* Formatting fixes
* Clang
* Slight reduction of user space
Still large enough to handle EA's shenanigans, but small enough that Linux doesn't break.
* Assert on VirtualQuery failure
* Extra debugging information
* Further reduce user space
This will unfix most of EA's titles, but UFC will still work.
Older windows versions support the high addresses, but trying to actually use them causes significant performance issues.
* Extra debugging info
Just in case other testers still run into issues.
* Remove debug logging
* Revert user space increases
Technically this constant is still higher than before, but weird side effects of our old logic resulted in a max address somewhere around this in main.
* address_space: Support expanded virtual memory space on macOS.
Co-Authored-By: squidbus <175574877+squidbus@users.noreply.github.com>
* Move address space constants to address_space.cpp
This ensures that all code must use the calculated address space memory values instead of the constants, since the calculated values can differ based on the platform.
This does require slight modification to thread state and gnmdriver code, since both were already using these constants directly.
* Workaround Windows 10 limitations
If a Windows 10 device is detected, use a lower value for USER_MAX to prevent system-wide hangs in VirtualAlloc2 calls.
* Fix compile for Windows-Qt
* Move tessellation_factors_ring_addr initialization to sceGnmGetTheTessellationFactorRingBufferBaseAddress
* Set image base address on Windows
This seems to work fine on Windows 11, needs testing from Windows 10 due to the previously discussed bugs.
* Set Linux executable base to 0x700000000000
This allows Linux to map the full user space without any workarounds.
Co-Authored-By: Marcin Mikołajczyk <2052578+mikusp@users.noreply.github.com>
* Basic formatting changes
* Reduce USER_MAX on Linux
Seems like finding a reliable way to move shadPS4's position in memory is difficult, for now limit the user size so we aren't trying to overwrite ourselves.
* Move memory and address_space variables.
---------
Co-authored-by: squidbus <175574877+squidbus@users.noreply.github.com>
Co-authored-by: Marcin Mikołajczyk <2052578+mikusp@users.noreply.github.com>
* Add configurable extra memory
* lowercase getter and setter
* Refactor memory setup to configure maximum memory limits at runtime
* sir clang offnir, the all-formatting
* Correctly update BackingSize on W*ndows too
* small format change
* remove total_memory_to_use from the header
* i have no idea how to name this commit
"addressing review comments" is a good name i guess
* Do not include extraDmem in the general config
* Fix flag handling on Windows
Fixes a weird homebrew kalaposfos made
* Fix backing protects
Windows requires that protections on areas committed through MapViewOfFile functions are less than the original mapping.
The best way to make sure everything works is to VirtualProtect the code area with the requested protection instead of applying prot directly.
* Fix error code for sceKernelMapDirectMemory2
Real hardware returns EINVAL instead of EACCES here
* Fix prot setting in ProtectBytes
* Handle some extra protection-related edge cases.
Real hardware treats read and write as separate perms, but appends read if you call with write-only (this is visible in VirtualQuery calls)
Additionally, execute permissions are ignored when protecting dmem mappings.
* Properly handle exec permission behavior for memory pools
Calling sceKernelMemoryPoolCommit with executable permissions returns EINVAL, mprotect on pooled mappings ignores the exec protection.
* Clang
* Allow execution protection for direct memory
Further hardware tests show that the dmem area is actually executable, this permission is just hidden from the end user.
* Clang
* More descriptive assert message
* Align address and size in mmap
Like most POSIX functions, mmap aligns address down to the nearest page boundary, and aligns address up to the nearest page boundary.
Since mmap is the only memory mapping function that doesn't error early on misaligned length or size, handle the alignment in the libkernel code.
* Clang
* Fix valid flags
After changing the value, games that specify just CpuWrite would hit the error return.
* Fix prot conversion functions
The True(bool) function returns true whenever value is greater than 0. While this rarely manifested before because of our wrongly defined CpuReadWrite prot, it's now causing trouble with the corrected values.
Technically this could've also caused trouble with games mapping GpuRead permissions, but that seems to be a rare enough use case that I guess it never happened?
I've also added a warning for the case where `write & !read`, since we don't properly handle write-only permissions, and I'm not entirely sure what it would take to deal with that.
* Fix some lingering dmem issues
ReleaseDirectMemory was always unmapping with the size parameter, which could cause it to unmap too much. Since multiple mappings can reference the same dmem area, I've calculated how much of each VMA we're supposed to unmap.
Additionally, I've adjusted the logic for carving out the free dmem area to properly work if ReleaseDirectMemory is called over multiple dmem areas.
Finally, I've patched a bug with my code in UnmapMemory.
* Remove mapped dmem type
Since physical addresses can be mapped multiple times, tracking mapped pages is not necessary.
This also allows me to significantly simplify the MapMemory physical address validation logic.
* Proper implementation for sceKernelMtypeprotect
I've rewritten SetDirectMemoryType to use virtual addresses instead of physical addresses, allowing it to be used in sceKernelMtypeprotect.
To accommodate this change, I've also moved address and size alignment out of MemoryManager::Protect
* Apply memory type in sceKernelMemoryPoolCommit
* Organization
Some potentially important missing mutexes, removed some unnecessary mutexes, moved some mutexes after early error returns, and updated copyright dates
* Iterator logic cleanup
Missing end check in ClampRangeSize, and adjusted VirtualQuery and DirectMemoryQuery.
* Clang
* Adjustments
* Properly account for behavior differences in MapDirectMemory2
Undid the changes to direct memory areas, added more robust logic for changing dma types, and fixed DirectMemoryQuery to return hardware-accurate direct memory information in cases where dmas split here, but not on real hardware.
I've also changed MapMemory's is_exec flag to a validate_dmem flag, used to handle alternate behavior in MapDirectMemory2. is_exec is now determined by the use of MemoryProt::CpuExec instead.
* Clang
* Add execute permissions to physical backing
Needed for executable mappings to work properly on Windows, fixes regression in RE2 with prior commit.
* Minor variable cleanup
* Update memory.h
* Prohibit direct memory mappings with exec protections
Did a quick hardware test to confirm, only seems to be prohibited for dmem mappings though.
* Update memory.cpp
* Implement sceKernelMemoryPoolGetBlockStats
Not entirely sure on the logic behind the cached blocks work, but flushed blocks seems to just be based on committed direct memory.
* Fix comment
* Refactor direct memory areas
At this point, swapping the multiple booleans for an enum is cleaner, and makes it easier to track the state of a direct memory area.
I've also sped up the logic for mapping direct memory by checking for out-of-bounds physical addresses before looping, and made the logic more solid using my dma type logic.
* Fix PoolCommit assert
Windows devices will throw an access violation if we don't check for iterator reaching end.
* Fix isDevKit
Previously, isDevKit could increase the physical memory used above the length we reserve in the backing file.
* Physical backing for flexible allocations
I took the simple approach here, creating a separate map for flexible allocations and pretty much just copying over the logic used in the direct memory map.
* Various fixups
* Fix mistake #1
* Assert + clang
* Fix 2
* Clang
* Fix CanMergeWith
Validate physical base for flexible mappings
* Clang
* Physical backing for pooled memory
* Allow VMA splitting in NameVirtualRange
This should be safe, since with the changes in this PR, the only issues that come from discrepancies between address space and vma_map are issues related to vmas being larger than address space mappings. NameVirtualRange will only ever shrink VMAs by naming part of one.
* Fix
* Fix NameVirtualRange
* Revert NameVirtualRange changes
Seems like it doesn't play nice for Windows
* Clean up isDevKit logic
We already log both isNeo and isDevKit in Emulator::Run, so the additional logging in MemoryManager::SetupMemoryRegions isn't really necessary.
I've also added a separate constant for non-pro devkit memory, as suggested.
Finally I've changed a couple constants to use the ORBIS prefix we generally follow here, instead of the SCE prefix.
* Erase flexible memory contents from physical memory on unmap
Flexible memory should not be preserved on unmap, so erase flexible contents from the physical backing when unmapping.
* Expand flexible memory map
Some games will end up fragmenting the physical backing space used for flexible memory. To reduce the frequency of this happening under normal circumstances, allocate the entirety of the remaining physical backing to the flexible memory map.
This is effectively a workaround to the problem, but at the moment I think this should suffice.
* Clang
* Fix address formatting for invalid address asserts
* Reduce event flag message to trace
This log effectively contributes nothing to debugging.
* Remove excess logging in sceKernelBatchMap
Every one of these opcodes will just log from their individual function calls anyway, and if there were issues with batch map logic, we would've known for a while now.
* Log error return during sceKernelBatchMap
May help with debugging some unstable UE games?
Somehow I missed that we already had a IsValidAddress function that could be used for ensuring addresses is inside vma map (for non Mac platforms at least). This PR swaps my Contains(addr) checks for calls to IsValidAddress.
This should help with some weird inconsistent memory asserts and exceptions caused by inconsistent iterator behavior on Windows devices.
* Swap !IsFree() for IsMapped()
IsFree only checks if the VMAType == Free. As is, that means ClampRangeSize will include memory that is Reserved or PoolReserved, and neither of those types are GPU mapped.
This fixes this bug, may help with some non-GPU memory asserts.
* Apply ClampRangeSize to vertex buffers
Helps with some cases encountered by UE and Minecraft.
* Properly align address and size in munmap
Based on observations of FreeBSD source code, fixes a Windows-related memory issue in War Thunder (CUSA00224)
* Format len and phys_addr in mmap
This should make logs slightly easier to understand, since we format these parameters in other memory calls.
* Update memory.cpp
* Validate requested dmem range in MapMemory
Handles a rare edge case that only comes up when modding Driveclub
* Specify type
auto has failed us once again.
* Types cleanup
Just some basic tidying up.
* Clang
* Merge dmem areas
* Fix DirectMemoryArea::CanMergeWith
Don't merge dmem areas if the memory types are different.
* Reduce some warnings to info
Both functions should behave properly now, there's no reason to warn about their use.
* Clang
* texture_cache: Avoid gpu tracking assert on sparse image
At the moment just take the easy way of creating the entire image normally and uploading unmapped subresources are zero
* tile_manager: Downgrade assert to error
* fix macos
* Swap do-while to while
If we use a do-while loop, we waste time if `aligned_size = 0`. This is also still accurate to FreeBSD behavior, where it returns success if `start == end` during mprotect.
This also effectively prevents the memory assert seen in updated versions of RESIDENT EVIL 2 (CUSA09193)
* Move prot validation outside loop
The prot variable shouldn't change during a mprotect call, so we can check the flags before protecting instead.
Also cleans up the code for prot validation.
This should improve performance, and is more accurate to FreeBSD code.
* Add logging for protect calls
This will help in debugging future problems
* Only perform GPU memory mapping when GPU can access it
This better aligns with hardware observations, and should also speed up unmaps and decommits, since they don't need to be compared with the GPU max address anymore.
* Reserve fixes
ReserveVirtualRange seems to follow the 0x200000000 base address like MemoryPoolReserve does.
Both also need checks in their flags Fixed path to ensure we're mapping in-bounds. If we're not in mapping to our address space, we'll end up reserving and returning the wrong address, which could lead to weird memory issues in games.
I'll need to test on real hardware to verify if such changes are appropriate.
* Better sceKernelMmap
Handles errors where we would previously throw exceptions. Also moves the file logic to MapFile, since that way all the possible errors are in one place.
Also fixes some function parameters to align with our current standards.
* Major refactor
MapDirectMemory, MapFlexibleMemory, ReserveVirtualRange, and MemoryPoolReserve all internally use mmap to perform their mappings. Naturally, this means that all functions have similar behaviors, and a lot of duplicate code.
This add necessary conditional behavior to MapMemory so MemoryPoolReserve and ReserveVirtualRange can use it, without disrupting the behavior of MapDirectMemory or MapFlexibleMemory calls.
* Accurate phys_addr for non-direct mappings
* Properly handle GPU access rights
Since my first commit restricts GPU mappings to memory areas with GPU access permissions, we also need to be updating the GPU mappings appropriately during Protect calls too.
* Update memory.cpp
* Update memory.h
* Update memory.cpp
* Update memory.cpp
* Update memory.cpp
* Revert "Update memory.cpp"
This reverts commit 2c55d014c0.
* Coalesce dmem map
Aligns with hardware observations, hopefully shouldn't break anything since nothing should change hardware-wise when release dmem calls and unmap calls are performed?
Either that or Windows breaks because Windows, will need to test.
* Implement posix_mprotect
Unity calls this
Also fixes the names of sceKernelMprotect and sceKernelMtypeprotect, though that's more of a style change and can be reverted if requested.
* Fix sceKernelSetVirtualRangeName
Partially addresses a "regression" introduced when I fixed up some asserts.
As noted in the code, this implementation is still slightly inaccurate, as handling this properly could cause regressions on Windows.
* Unconditional assert in MapFile
* Remove protect warning
This is expected behavior, shouldn't need any logging.
* Respect alignment
Forgot to properly do this when updating ReserveVirtualRange and MemoryPoolReserve
* Fix Mprotect on free memory
On real hardware, this just does nothing. If something did get protected, there's no way to query that information.
Therefore, it seems pretty safe to just behave like munmap and return size here.
* Minor tidy-up
No functional difference, but looks better.
* Properly handle ENOMEM error return in MapMemory
Needed for Assassin's Creed Unity to behave properly.
* Change error message
If I left the message as-is, we'd probably see inexperienced people claiming the assert means your device needs more memory, which is completely false.
* Clang
You know you're doing something right when Clang complains.
* Attempt to handle MemoryMapFlags::NoOverwrite
Based on my interpretation of red_prig's descriptions. These changes are untested, as I'm not able to test right now.
* Fix flag description
* Move overwrite check to while condition
* Implement sceKernelMapDirectMemory2
Behaves similarly to sceKernelMapDirectMemory, but has a type parameter.
* Simplify
No need to copy all the MapDirectMemory code over, can just call the function, then do the SetDirectMemoryType call
* Clang
* Update sceKernelMemoryPoolExpand
Hardware tests show that this function is basically the same as sceKernelAllocateDirectMemory, with some minor differences.
Update the memory searching code to match my updated AllocateDirectMemory code, with appropriate error conditions.
* Update MemoryPoolReserve
Only difference between real hw and our code is behavior with addr = 0.
* Don't coalesce PoolReserved areas.
Real hardware doesn't coalesce them.
* Update PoolCommit
Plenty of edge case behaviors to handle here.
Addresses are treated as fixed, EINVAL is returned for bad mappings, name should be preserved from PoolReserving, committed areas should coalesce, reserved areas get their phys_base updated
* Formatting
* Adjust fixed PoolReserve path
Hardware tests suggest this will overwrite all VMAs in the range. Run UnmapMemoryImpl on the full area, then reserve. Same logic applies to normal reservations too.
Also adjusts logic of the non-fixed path to more closely align with hardware observations.
* Remove phys_base modifications
This can be handled later. Doing the logic properly would likely take work in MergeAdjacent, and would probably need to be applied to normal dmem mappings too.
* Use VMAHandle.Contains()
Why do extra math when we have a function specifically for this?
* Update memory.cpp
* Remove unnecessary code
Since I've removed those two asserts, these two lines of code effectively do nothing.
* Clang
* Fix names
* Fix PoolDecommit
Should fix the address space regressions in UE titles on Windows.
* Fix error log
Should make the cause of this clearer?
* Clang
* Oops
* Remove coalesce on PoolCommit
Windows makes this more difficult.
* Track pool budgets
If you try to commit more pooled memory than is allocated, PoolCommit returns ENOMEM.
Also fixes error conditions for PoolDecommit, that should return EINVAL if given an address that isn't part of the pool.
Note: Seems like the pool budget can't hit zero? I used a <= comparison based on hardware tests, otherwise we're able to make more mappings than real hardware can.
* Fix VirtualQuery behavior on low addresses.
* Fix VirtualQuery struct
Somewhere in our BitField and array use, the size of our VirtualQuery struct became larger than the struct used on real hardware.
Fixing this fixes some data corruption visible in the name parameter during my tests.
* Default name to anon
On real hardware, nameless mappings are given the name "anon:address" where address appears to be the address that made the memory call.
For simplicity sake, I'll stick to the name "anon" for now.
* Place an upper bound on returns from SearchFree
Right now, this upper bound is set based on the limitations of our GPU buffer cache and page table.
Someone with more experience in that area of code should probably fix that at some point.
* More anons
* Clang
* Fix name in sceKernelMapNamedDirectMemory
* strncpy instead of strcpy
Hardcoded the constant size for now, I need to review how real hardware behaves here to determine if anything else is necessary for this to be accurate.
* Fix name behavior
All memory naming functions restrict the name size to a 31 character limit, and return `ORBIS_KERNEL_ERROR_ENAMETOOLONG` if that limit is exceeded.
Since this value is constant for all functions involving names, I've defined it as a constant in kernel's memory.h, and used that in place of any hardcoded 32 character limits.
* Error logging
Hopefully this helps in catching the UFC regression?
* Increase address space upper bound
Probably needs heavy testing, especially on Mac/Windows.
This increases the address space, as needed to accommodate strange memory behaviors seen in UFC.
* VirtualQuery fix
Due to limitations of certain platforms, we initialize our vma_map with 3 separate free mappings.
As such, we need to use a while loop here to accurately query mappings with high addresses
* Fix mappings to high addresses
The PS4's GPU can only handle 40bit addresses. Our texture cache and buffer cache were designed around these limits, and mapping to higher addresses would cause segmentation faults and access violations.
To fix these crashes, only map to the GPU if the mapping is fully contained within the address space the GPU should access.
I'm open to suggestions on how to make this cleaner
* Revert "Increase address space upper bound"
This reverts commit 3d50eeeebb.
* Revert VirtualQuery while loop
Windows wasn't happy with this, again.
Will try to debug and properly fix this when I have a good chance.
* Fix asserts
FindVMA, due to the way it's programmed, never actually returns vma_map.end(), the furthest it ever returns is the last valid memory area. All those asserts we involving vma_map.end() never actually trigger due to this.
This commit removes redundant asserts, adds messages to asserts that were lacking them, and fixes all asserts designed to detect out of bounds memory accesses so they actually trigger.
I've also fixed some potential memory safety issues.
* Proper error behavior in QueryProtection
Might as well handle this properly while I'm here.
* Clang
* More information about ReserveVirtualRange results
Should help debug issues like the one in The Order: 1886 (CUSA00076)
* Fix assert message
* Update assert message
Extra space
* Fix my bug
Oh hey, finally something that's my fault.
* Fix rasterizer unmaps
Should use adjusted_size here, otherwise we could unmap too much.
Thanks to diegolix29 for spotting this.
* Fix edge case in MapMemory
Code comments explain everything.
This should fix some memory asserts.
* Fix fix
Avoid running the code path if it's unnecessary, since there are many additional edge cases to handle when the VMA map is small.
* Fix fix fix
Should prevent infinite loops, haven't tested properly yet though.
* Split logging for inputs and out_addr in ReserveVirtualRange
Addresses review comments.
* Update memory.cpp
* Clean logic
FindDmemArea guarantees that the first dmem area we check contains search_start. Any dmem areas beyond the first one will be entirely past search_start, so checking against it in the loop is unnecessary.
* Emulate memory behavior of libSceGnmDriver _DT_INIT
Due to the unique way some games check for sceKernelAllocateDirectMemory failures, emulating this properly is necessary.
* Clang
* Fix address input for direct memory call
* Fix bug with DirectQueryAvailable
Missed this in my prior PR.
* DirectQueryAvailable fix
Fixes error cases to be more hardware accurate.
* AllocateDirectMemory fixes
search_start and search_end were ignored in certain cases, this fixes that issue.
I've also basically rewritten the function in the process, since the lack of documentation made it difficult to make the proper adjustments.
* DirectQueryAvailable fixes
remaining_size was calculated incorrectly in cases where a free dmem_area had a base earlier than search_start, or an end after search_end.
* Reduce sceKernelGetDirectMemorySize log severity
By this point, we've confirmed that sceKernelGetDirectMemorySize is hardware-accurate. There's no reason to clog logs with this function, which games usually call before every sceKernelAllocateDirectMemory call.
* Clang
* Fix phys_addr_out
phys_addr_out should be equal to search_start in cases where search_start is greater than the dmem_area base.
* Dividing by zero is fun
Need to check for alignment when aligning things.
* Update memory.cpp
* Clang
* ir_passes: Add barrier at end of block too
* vk_platform: Always assign names to resources
* texture_cache: Better overlap handling
* liverpool: Avoid resuming ce_task when its finished
* spirv_quad_rect: Skip default attributes
Fixes some crashes
* memory: Improve buffer size clamping
* liverpool: Relax binary header validity check
* liverpool: Stub SetPredication with a warning
* Better than outright crash
* emit_spirv: Implement round to zero mode
* liverpool: queue::pop takes the front element
* image_info: Remove obsolete assert
The old code assumed the mip only had 1 layer thus a right overlap could not return mip 0. But with the new path we handle images that are both mip-mapped and multi-layer, thus this can happen
* tile_manager: Fix size calculation
* spirv_quad_rect: Skip default attributes
---------
Co-authored-by: poly <47796739+polybiusproxy@users.noreply.github.com>
Co-authored-by: squidbus <175574877+squidbus@users.noreply.github.com>
* Implement protecting multiple VMAs
A handful of games expect this to work, and updated versions of Grand Theft Auto V crash if it doesn't work.
* Clang
* memory: Consider flexible mappings as gpu accessible
Multiple guest apps do this with perfectly valid sharps in simple shaders. This needs some hw testing to see how it is handled but for now doesnt hurt to handle it
* memory: Clamp large buffers to mapped area
Sometimes huge buffers can be bound that start on some valid mapping but arent fully contained by it. It is not reasonable to expect the game needing all of the memory, so clamp the size to avoid the gpu tracking assert
* clang-format fix
---------
Co-authored-by: georgemoralis <giorgosmrls@gmail.com>
* Handle errors in sceKernelAllocateDirectMemory
* Improve accuracy of error cases
Some of our existing cases are normally EAGAIN returns.
* Improve logging on errors
* Clang
* TEMPORARY HACK FOR NBA TESTS
This will be removed before this PR is marked as ready, and is only here to make sure the other NBA games (and perhaps DOA3) work if some missing init behavior is handled.
* Revert "TEMPORARY HACK FOR NBA TESTS"
This reverts commit a0e27b0229.
* Change error message
* Unmap memory in chunks if spanning over multiple VMAs
* clang
* Merge fixups
* Minor code style changes
* Update function declarations
---------
Co-authored-by: Marcin Mikołajczyk <marcinmikolajcz@gmail.com>
