8364107: [lworld] Revise memory limit in compiler/startup/StartupOutput.java

[marc-chevalier]

Let's do a similar job as in #1509 and see what evolved since JDK-8350209: Preserve adapters in AOT cache landed in Valhalla.

For each tick in the list, I give a range that behaves homogeneously, and compare it to a Mainline and/or Valhalla range of #1509

• [4, 24] ~ Mainline [4, 28]

  # There is insufficient memory for the Java Runtime Environment to continue.
  # Native memory allocation (malloc) failed to allocate 3000 bytes. Error detail: `CodeCache: no room for StubRoutines` (continuation stubs)
  

• [25, 28] ~ Mainline [29, 32] | Valhalla [29, 32]
  fatal error: Initial size of CodeCache is too small

• [29, 1164] ~ Mainline [33, 1112] | Valhalla [33, 1164]

  # There is insufficient memory for the Java Runtime Environment to continue.
  # Native memory allocation (malloc) failed to allocate 1108928 bytes. Error detail: CodeCache: no room for Interpreter
  

• [1165, 1172] ~ Mainline [1117, 1128] | Valhalla [1165, 1172]
  fatal error: Initial size of CodeCache is too small

• [1173, 1176] ~ Valhalla [1225, 1228] (out of order, weird, uh!)
  SIGSEGV in CodeSection::emit_int8(unsigned char)

• [1177, 1224] ~ Mainline [1129, 1144] | Valhalla [1173, 1220]
  assert(_buffer != nullptr) failed: should be initialized

• [1225, 1228] ~ Mainline [1145, 1148] | Valhalla [1221, 1224]
  assert(_no_arg_handler != nullptr && _obj_arg_handler != nullptr && _int_arg_handler != nullptr && _obj_int_arg_handler != nullptr && _obj_obj_arg_handler != nullptr) failed: Initial adapter handlers must be properly created

• [1229, ~1250] ~ Mainline [1149, ~1700]
  Often works, sometimes

  java.lang.OutOfMemoryError: Out of space in CodeCache for adapters
  

• [~1250, 1276]
  Mostly works

• [1277, 1432]

  java.lang.OutOfMemoryError: Out of space in CodeCache for adapters
  

• [1433, 1438] ~ Valhalla [1301, 1308]

  java.lang.InternalError: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToStatic(Object,Object,MemberName)Object/invokeStatic
  Caused by: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToStatic(Object,Object,MemberName)Object/invokeStatic
  

• [1439, ~1600]
  works but

  CodeCache is full. Compiler has been disabled.
  

• Above 1600, compiler shutdown starts to be rare, but again around 2800k Like it was for Valhalla before above 1994k, and we saw it around 3700k.

  It used to be:

  [1994, ...] mostly works, but sometimes the same error as above, witnessed until 3780k (and especially between 3640 and 3780). Interestingly, at 3700k, we also see some fine runs, with, or without compiler shutdown. Starting 3800k, I never saw a failure, and starting 3900k, no compiler shutdown.

Overall, the behavior for small amount of memory is similar as in previous Valhalla and mainline. The behavior for larger amounts got much better, more similar to mainline where it mostly works from 1149k, and up, except in some pathological islands.

So, it looks good to reduce the 2000 in compiler/startup/StartupOutput.java (was increased from 800). So, I just tried... If locally, I could indeed make it run with quite smaller amount of memory, some more testing shows that 1300 is a better bound: it crashes significantly starting under 1200 but seems to pass all tests with 1300. That is still an increase from 800, but less radical than 2000.

Thanks,
Marc

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (1 review required, with at least 1 Committer)

Issue

• JDK-8364107: [lworld] Revise memory limit in compiler/startup/StartupOutput.java (Enhancement - P4)

Reviewers

• Christian Hagedorn (@chhagedorn - Committer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/valhalla.git pull/1749/head:pull/1749
$ git checkout pull/1749

Update a local copy of the PR:
$ git checkout pull/1749
$ git pull https://git.openjdk.org/valhalla.git pull/1749/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1749

View PR using the GUI difftool:
$ git pr show -t 1749

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/valhalla/pull/1749.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back mchevalier! A progress list of the required criteria for merging this PR into lworld will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@marc-chevalier This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8364107: [lworld] Revise memory limit in compiler/startup/StartupOutput.java

Reviewed-by: chagedorn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 134 new commits pushed to the lworld branch:

• 405db7a: 8367151: [lworld] CorrectlyRestoreRfp.java triggers "bad oop found" during deoptimization
• a483e8c: 8209554: [lworld] ClassCastException thrown for JCK test instead of expected IllegalArgumentException
• c73220a: 8372345: [lworld] Problem list JDK-8372341
• ... and 131 more: https://git.openjdk.org/valhalla/compare/b1d14c658511cced2a860d9e2741ae89827e25ba...lworld

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the lworld branch, type /integrate in a new comment.

[marc-chevalier]

/touch

Trying to see if the bot find the issue this time.

[openjdk]

@marc-chevalier The pull request is being re-evaluated and the inactivity timeout has been reset.

[mlbridge]

Webrevs

• 00: Full (896eb630)

[chhagedorn]

Looks reasonable, thanks for the thorough analysis!

[marc-chevalier]

/integrate

Thanks @chhagedorn for review. I've run some tests again to make sure it didn't pass only by chance. Still looks good.

[openjdk]

Going to push as commit 0a86ffe.
Since your change was applied there have been 138 commits pushed to the lworld branch:

• 23f0b1e: 8372529: [lworld] sources/TestIncludesAreSorted.java failures caused by JDK-8371915
• fcffef4: 8371915: [lworld] LayoutKind enum should have helper methods
• 5ab1179: 8372394: [lworld] c1_LIRGenerator.cpp(2313): warning C4267: 'argument': conversion from 'size_t' to 'int'
• ... and 135 more: https://git.openjdk.org/valhalla/compare/b1d14c658511cced2a860d9e2741ae89827e25ba...lworld

Your commit was automatically rebased without conflicts.

[openjdk]

@marc-chevalier Pushed as commit 0a86ffe.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.