[Bug] probably a bug in the new parpavm optimizations

[ddyer0]

Summary

Well tested and unchanged dictionary loading code, worked everywhere on june 29 builds,
fails on todays IOS builds.

Minimal reproducer

run the test program. After loading 133000 of the definitions, it disappears somewhere and
never comes back.

Steps to reproduce

run the test program

Expected and observed behavior

should finish after a few seconds "loaded 279496 definitions"

actually get stuck at about 118000 definitions, but after a few very long pauses it finishes.
some kind of death by GC?

Codename One version

current

JDK version

java 16

Target environment

IOS fails

Last known working version

before parpavm upgrades

Logs and diagnostics

Screenshots or recordings

Dtest.java

dictionary.zip

unpack zip file to get the dictionary .res

sorry, there's a lot of boilerplate crudely ripped from my environment, but the point of interest
is in "loaddefinitions"

Report checks

• I searched existing issues and discussions for this behavior.
• The reproducer contains no credentials, signing keys, or private customer data.
• I understand that triage is not a commitment to a fix date.

[shai-almog]

Is this just a performance issue?

The way you're acting it sounded like a crash and words like "death by GC" imply that. Looking at the code you're clearly thrashing the GC and reaching its limits. Just don't do that.

[ddyer0]

It's a performance issue so severe that IOS reports the app crashed.
The main GUI has to wait for the dictionary to load, and the load time is way too long.

I was surprised that the dtest version eventually finished, but the GC profile is visibly awful in any case.

The pre-improvement parpavm had no trouble with the code, it ran to completion
in a few seconds. I've been looking at it (it's not mine) and it is profligate with string.getBytes()
and new ByteArrayOutputStream. Not hard to reduce the mutator profile.

[shai-almog]

This doesn't reproduce in any significant way on our side. But looking at the code I'm really not surprised. You reinvented UTF, streams etc. None of these are a good idea and this thrashes VM ram while you're doing it. Our UTF reading is SIMD optimizaed and is highly performant. Strings can produce a lot of garbage when used incorrectly as there are MANY subtle nuances in the core implementation you can't be aware of.

Looking at the code I have no idea why you did any of that instead of just using the Codename One APIs.

Instead of focusing on trying to revert and keeping the broken code chugging along I suggest you replace it with simple Codename One calls and get an instant fix.

[ddyer0]

after a first tranche of de-thrashing, my ipad finishes the load in 307 seconds. my tablet finishes
in 7 seconds. before, they were comparable.

[ddyer0]

for my purposes, the biggest premium is on using exactly the same code on java desktop and codenameone mobiles.
Also with exactly the behavior and interfaces I need.

[shai-almog]

I'm looking into this, but this by no means justifies the the type of public escalation you're driving. GC changes in Java SE make code slower all the time. Performance is about choices and we can't possibly benchmark every usage. In your case the code practically attacks our performance optimizations. I've been telling you for years that you're using Codename One while trying to pretend that you aren't, this ends up with issues like this experienced only/mostly by yourself.

Hotspot has the magic ability to undo an optimization that failed, we can't do that.

[shai-almog]

I'm closing it as a won't fix. The fact that it somehow worked in the past is a miracle. On Android/Java SE the JIT worked around it. In the old VM you just filled up gigabytes of RAM which were then collected later and chocked everything. It might have looked better but the cost was paid just non-deterministically.

• In this test case the loader first retains roughly 280,000 Entry objects, key strings, and Hashtable nodes.

• For every one of 279,496 definitions, readToWhitespace() creates a new temporary lookup String.

• Smaz.compress() then creates two new ByteOutputStream objects, two initial byte[32] buffers, possible expansion buffers, and one retained result array.

• That is at least six small allocations per definition: over 1.6 million small objects/arrays, excluding expansions and Hashtable work.

• Every definition is also retained as a separate tiny byte[], making subsequent GC scans progressively more expensive.

There is also a real bug in the Smaz loop. After finding a match it advances index, but breaks the inner while; it continues trying shorter lengths using hashes calculated for the old index. That adds several useless codebook scans per match and sometimes processes additional input using stale state. It is CPU amplification layered on top of the allocation problem.

The custom infrastructure contributes further problems:

• The hand-written UTF-8 decoder is incorrect for continuation bytes and four-byte sequences. As I mentioned before it is also DEEPLY inefficient compared to our SIMD implementation

• ResourceBundle.isData() calls Resources.getData(), allocating a stream, then getResourceAsStream() calls it again. That is wasteful but only startup noise.

• readToken() creates a new StringBuilder, String, and lowercase result for every dictionary word.

• The code claims to avoid string thrashing, then replaces it with substantially worse tiny-array and tiny-stream thrashing.

• Storing 280,000 definitions as separate arrays is inherently hostile to tracing GC. A contiguous compressed blob plus integer offsets would be dramatically cheaper.

The only way to change that behavior is to essentially make every other behavior worse. That would make sense if this issue made sense and represented an actual problem. What I see here is GC abuse.

[ddyer0]

Thanks for the analysis, I have a slightly better understanding now. If this had never worked well, I think your explanation
of the inefficiencies would be persuasive. In this case though, it always worked adequately and somehow your optimizations
made it behave exponentially worse. Given that the algorithm and sequence of garbage creation is identical, I crave an
explanation how your optimizations made such a large difference.

In a more global context, this is (or was) an incidental piece of code that never showed up as needing attention. Suddenly
it has become a critical issue. Other users may have similar issues.

[ddyer0]

re: What I see here is GC abuse.

A big part of the philosophy and argument in favor of GC memory management is that short lived garbage
is easily taken care of. I've always been skeptical of that, too.