[DYNAREC] Add BOX64_DYNAREC_LDACQ to apply STRONGMEM=4 to selected ranges

[Ericky14]

DISCLAIMER: I have used AI to debug and research a possible solution for Cyberpunk crashing for me on XElite 2 hardware, as well as learning about the codebase.

Crash:

EXCEPTION_ACCESS_VIOLATION (0xC0000005)
The thread attempted to write to an inaccessible address at 0x0.

#0 Cyberpunk2077.exe+0x14221a 
#1 Cyberpunk2077.exe+0x14324c
#2 Cyberpunk2077.exe+0x16d50e
#3 Cyberpunk2077.exe+0x1a3abf4
#4 kernel32.dll+0x11649 

Example of how I am using it:

BOX64_DYNAREC_LDACQ=Cyberpunk2077.exe:142000-144000

Ranges desc:

RVA	                        function
142190-1422d9	generic ParallelFor executor (where it faulted)
143060-1435d7	job worker main loop + inline ring dequeue
14379c-1437f8	secondary ring pop helper

I tried STRONGMEM=4 and it made the launcher unusable FPS-wise.

[ksco]

Thanks for the debugging, but are you sure this fixes your issue? Because if it's related to strongmem emulation, it can be random sometimes.

I ask this because CP2077 is not an old game; it's compiled by modern MSVC and thus has volatile metadata, which Box64 utilizes to do strongmem emulation. The strongmem emulation for this game should already be perfect.

[ksco]

I prefer a BOX64_DYNAREC_STRONGMEM_RANGE, which can be used together with BOX64_DYNAREC_STRONGMEM.

[Ericky14]

I renamed it

[ksco]

This isn't just a simple rename. BOX64_DYNAREC_STRONGMEM_RANGE should only accept a range, not the exe name.

[ptitSeb]

I need to study the code in this interval to see why volatile metadata would not be enough

[Ericky14]

Thanks for the debugging, but are you sure this fixes your issue? Because if it's related to strongmem emulation, it can be random sometimes.

I ask this because CP2077 is not an old game; it's compiled by modern MSVC and thus has volatile metadata, which Box64 utilizes to do strongmem emulation. The strongmem emulation for this game should already be perfect.

I was experiencing the game freezing a little bit randomly usually 1-5 minutes into the game when you loaded a save at the very beginning part of the game, it was a specific issue because it didn't even show a crash dialog. Since the fixes I haven't been able to reproduce it.

It could also be that this is brand new hardware and exposing a bug not seen before either. I was trying to get the game stable for a demo.

If you have any debugging steps you'd like me to run through to ensure this is the correct fix please let me know, I'm sure you are much more experienced with box64 than I am.

Just to clarify as well, this is the Epic Games Cyberpunk, not Steam, in case they are different in any way.

[ksco]

I need to study the code in this interval to see why volatile metadata would not be enough

@Ericky14 What @ptitSeb said is what I imply; we need to understand why, as this is not expected. Anyway, good work!

[ptitSeb]

Well, after dumping the Volatile Metadata informations: the range than Ericky found is out of the volatile metadata ranges!

[ksco]

I am wondering if it would be a more reasonable choice to provide a mechanism for adding supplementary volatile metadata on top of the original.

[ptitSeb]

I am wondering if it would be a more reasonable choice to provide a mechanism for adding supplementary volatile metadata on top of the original.

It would be a nice addition for sure, but it would require a lot of work to identify those data, and also would be dependant on the version of the game, wich we do not test at any time.

[ptitSeb]

@Ericky14 side question: is the STRONGMEM=4 needed for this range, or does a lower strongmem settings, like 2 or 3, would be enough?

[ksco]

I am wondering if it would be a more reasonable choice to provide a mechanism for adding supplementary volatile metadata on top of the original.

It would be a nice addition for sure, but it would require a lot of work to identify those data, and also would be dependant on the version of the game, wich we do not test at any time.

Indeed.

[Ericky14]

@Ericky14 side question: is the STRONGMEM=4 needed for this range, or does a lower strongmem settings, like 2 or 3, would be enough?

I set global 3, and 4 for this specific range.
I reproduced the crash with global value set to 3.

We don’t need all features of 4 for this range, might be able to get away with a different design covering only the load fences for proper fix

[ptitSeb]

Wow, 3 is already super strong, and put many many fences, while 4 just pace fences on all memory access. It's very strange that such strong settings s needed on X2-Elite architecture while nothing specific is needed for others arch. I need to understand why there is such a difference in behaviour, because 3 eats already a lot of speed, espcialy compared to 0!

[ptitSeb]

We don’t need all features of 4 for this range, might be able to get away with a different design covering only the load fences for proper fix

What do you mean by that, have you identify exactly where fences should be added in the code?

[airidosas252]

I personally find it hard to believe that another STRONGMEM option is needed here.

Technically speaking volatile metadata here should be enough to emit barriers in the right places to avoid crashes.

Unless you are using older builds of Cyberpunk, that can explain why the game crashes. New versions do carry volatile metadata (version 2.3.1 for example I'm using for testing)

[ptitSeb]

I personally find it hard to believe that another STRONGMEM option is needed here.

Technically speaking volatile metadata here should be enough to emit barriers in the right places to avoid crashes.

Unless you are using older builds of Cyberpunk, that can explain why the game crashes. New versions do carry volatile metadata (version 2.3.1 for example I'm using for testing)

I use latest Cyberpunk version from GoG, and Volatile metadata coverarge have a huge hole:

info range table: 0x03158ee8 (6 entries)
  range[0]: RVA=0x0000cc70 size=0x18f0
  range[1]: RVA=0x019657f0 size=0x13a0
  range[2]: RVA=0x01a3e270 size=0x1af220
  range[3]: RVA=0x01d93a60 size=0xd0
  range[4]: RVA=0x01d93d40 size=0x20
  range[5]: RVA=0x01d93fd0 size=0x5d0

unless we parse the matadata incorectly or some other tables are available elsewhere

[ksco]

No, it makes no sense to support module: syntax. Please take a look at USAGE.md; see how the RC file works.

[ptitSeb]

I personaly like the module syntax! It make sense to me, as many module can be oad at different addresses, and having a module+offset is a nice to have feature, that I would like to see on other settings that use address ranges

[ksco]

Do we agree that this is a rarely used option, probably only in Cyberpunk 2077? If we agree, I would like

[Cyberpunk2077.exe]
BOX64_DYNAREC_STRONGMEM_RANGE=0x142000-0x144000

instead of

BOX64_DYNAREC_STRONGMEM_RANGE=Cyberpunk2077.exe:142000-144000

[Ericky14]

I think we can keep this as a general debug option, I am trying to look deeper into the Cyberpunk issue to find the root cause and what really caused it, it probably isn’t a good idea to hard code the address range since it can change

[ptitSeb]

Do we agree that this is a rarely used option, probably only in Cyberpunk 2077? If we agree, I would like

[Cyberpunk2077.exe]
BOX64_DYNAREC_STRONGMEM_RANGE=0x142000-0x144000

instead of

BOX64_DYNAREC_STRONGMEM_RANGE=Cyberpunk2077.exe:142000-144000

To me, that's not the same things at all. The first range means absolute address (which, for this game, are probably not a avalid range as the main exe loads at 0x14000000000), while the second is a range of relative address, based on the load address of Cyberpunk2077.exe
The point is, we could also use a dll as the base address for a range. It can be really usefull when the exe or dll loads at a non-fixed address

[ksco]

Okay, I see. Then we should only keep this form: BOX64_DYNAREC_STRONGMEM_RANGE=Cyberpunk2077.exe:142000-144000, not both.