This repository was archived by the owner on Aug 25, 2026. It is now read-only.
Use a lot less memory, and give it back when the system asks - #85
Closed
alltechdev wants to merge 2 commits into
Closed
alltechdev wants to merge 2 commits into
alltechdev wants to merge 2 commits into
Conversation
Reported on a TCL Flip 2 as "the whole app is a bit slow" (#83). Measured on an M5 (2.9 GB, Android 13, standardDebug, median of 5 cold starts) the app held 831 MB at peak, sat at 421 MB idle, and gave back nothing at all when the OS asked it to shrink, because nothing in the tree implemented memory-pressure handling. The biggest win needs no device gate. The on-device speech model costs ~267 MB while loaded (~101 MB of weights plus ~146 MB of onnxruntime arena) and was kept for the whole process on the chance of a mic tap many users never make. It is now dropped after two minutes unused and rebuilt on next use, so the instant first tap that was asked for on 2026-07-10 is kept while the session-long hold is not. Device-verified: scudo:secondary 111 MB to 9 MB at the 120 s mark, model rebuilding correctly afterwards. Idle PSS 421 MB to 299 MB on a phone that is not low-RAM at all. Nothing released under pressure. There was no onTrimMemory, onLowMemory or ComponentCallbacks2 anywhere, so a TRIM_MEMORY_COMPLETE freed 0 KB and the system's only remaining option was to kill us. VelaApp now fans every trim out through a new MemoryPressure holder, and the things that actually hold memory register a release: the speech model, the neural voice, MapLibre's native tile and sprite caches (MapView.onLowMemory was never called), all five hidden WebViews, and the image cache. WhisperRecognizer had no release path at all, so even Remove-model left ~267 MB resident for the rest of the process. Nothing adapted to the device either. There was no isLowRamDevice branch and the image cache was a flat 48 MB whatever the phone. Constrained devices now skip the startup preload of the speech model, cap images at 16 MB, skip the speculative WebView warm on every search, and fetch 8 ambient POI category terms instead of 15 with a smaller result pool. Roomier phones keep their existing behaviour. The low-RAM POI subset deliberately keeps school and park: the ambient layer filter-hides the basemap OSM poi layers at z14+, so those two have no second source and a first 6-term subset made every park and school pin vanish. Caught by an A/B screenshot, not by any test. Also stop shipping x86 and x86_64 native libraries. No phone Vela targets can execute them and libmaplibre.so alone carried 23 MB of them into every install. armeabi-v7a stays, since 32-bit ARM keypad phones are real. Measured, main vs this, low-RAM path: peak 831 MB to 581 MB (-30%), post-trim 397 MB to 246 MB (-38%), native heap 223 MB to 95 MB (-57%), cold start 4811 ms to 4333 ms. On a normal-RAM device idle drops 29%, post-trim 28% and native heap 44%. APK 97.4 MB to 92.0 MB. Debug builds honour `setprop debug.vela.lowram true` so the low-RAM path can be exercised on a dev phone, where it is otherwise dead code (every device we own reports lowRam=false heapClassMb=256). AGENTS.md: document the seam and the measurement traps, and correct the memory rule, which said the OverpassTrafficSignals/OverpassPois stream-parse follow-up was pending. It has been done for some time; the remaining buffered hot reader is the Google ambient path, and chasing the stale line wasted a pass.
alltechdev
added a commit
that referenced
this pull request
Jul 21, 2026
PR #85 gave every big holder a release() and fanned OS trims out to them, but a Kotlin release only returns pages to scudo. They sit on its free lists, where RSS/PSS still count them and lmkd still sees a fat process. Measured on the M5 before this change: a full TRIM_MEMORY_COMPLETE with all 8 listeners firing moved scudo:primary 56,578 to 54,978 KB while mallinfo reported a 442 MB arena holding just 46 MB live. mallopt() is the only way to hand that gap on and it is reachable only from C, so this adds the app's first native module: app/src/main/cpp/velamem.cpp, three lines calling libc, 4 KB for arm64 and 2.7 KB for armeabi-v7a. Built for those two ABIs only, matching the x86 drop in #85. MemoryPressure.dispatch schedules it 750 ms after a trim, off the main thread. The delay is load-bearing: the WebView reapers post destroy() to the main looper and VelaApp clears Coil after dispatch returns, so an inline purge would run before the memory it is meant to reclaim had been freed. The purge fires from TRIM_MEMORY_RUNNING_LOW (10) up, deliberately wider than isSevere (40). Measured: pressing HOME delivers only TRIM_MEMORY_UI_HIDDEN (20), never BACKGROUND (40), so gating on isSevere would skip the single most common moment we are handed, the one where the app is off-screen and nothing can jank. Verified by A/B on ONE binary, since two builds also differ in background settling and idle PSS swings +-60 MB run to run. debug.vela.nopurge suppresses the purge at runtime, making the delta paired within a run; both arms were checked in logcat to confirm the gate actually gates. Releasing the ASR model, 3 alternating pairs: with the purge suppressed scudo:primary moved 60/32/28 KB in the 7 s after the trim, which is nothing, and with it on 3704/3008/2792 KB. No overlap. A second 8-pair A/B over map and POI churn agreed: 3345 to 6931 KB mean reclaimed, Mann-Whitney U=7 at n=8/8, p<0.05. It is worth a consistent ~3 MB, not tens, and the commit says so rather than claiming the onnxruntime arena. The ASR model's ~111 MB lives in scudo:secondary, which is mmap-backed and comes back on free() with no purge needed (111 MB to 7 MB in BOTH arms). Only scudo:primary needs asking. M_PURGE_ALL is API 34+, so on the Android 13 dev phone it returns 0 and the code falls back to M_PURGE (API 28+). The logged mode= says which actually took, so a device supporting neither is visible instead of silently doing nothing. Also corrects two things #85 left wrong. The MemoryPressure KDoc told the reader to run `setprop debug.vela.lowram ""` for real detection, which AGENTS.md itself says is a shell syntax error; and `false` does not restore detection either, it forces the normal path. Clearing needs an unparseable value, so both the KDoc and AGENTS.md now say `none`, which is verified: the app logs forced=no. AGENTS.md also gains the UI_HIDDEN-vs-BACKGROUND finding, the one-binary A/B rule, and the asr_model_bad trap: a quarantined model makes warmUp() a silent no-op, so scudo:secondary sits at ~11 MB instead of ~111 MB and a memory benchmark measures the model-absent case without saying so. That cost a run here. Verified: :app:detekt and :core:detekt 0 smells, :core:test green, audit_deadcode.sh PASS, standardDebug builds, installs, launches with no UnsatisfiedLinkError, and renders correctly on device (screenshot: map, POI pins including parks, focus ring, soft keys).
Conflict resolutions favor the fork's decisions where main deliberately diverged, and graft #85's memory work onto main's newer structure: - app/build.gradle.kts: keep main's jniLibs excludes (v7a stays - feature phones ship 32-bit userspace; only the ONNX/sherpa x86 copies drop). - WhisperRecognizer: keep main's multi-engine loader (loadedKey, per-engine strikes/quarantine) and add #85's idle reaper, severe-trim release and low-RAM warm-up skip on top. - MapViewModel: keep main's per-engine delete/select/onboarding flows; the release-before-delete from #85 moves into deleteAsrEngine.
alltechdev
added a commit
that referenced
this pull request
Jul 23, 2026
Same resolution stance as the #85 branch merge (437c24e), applied to this branch's rewrites: - WhisperRecognizer: keep this branch's lease/ReentrantLock release architecture (the use-after-free fix) and port main's multi-engine loader (engine+lang key, per-engine strikes/quarantine, three model configs) into ensureRecognizerLocked. loadedLang becomes loadedKey. - MapViewModel: keep main's per-engine flows; release(wait = true) moves into deleteAsrEngine so Remove still frees the native model first. - app/build.gradle.kts: keep main's ARM-only ndk abiFilters and ONNX-only packaging excludes; keep this branch's cmake abiFilters for libvelamem.
Owner
Author
|
Superseded by #86, which now contains all of this branch's commits merged with main (multi-engine ASR + ABI decisions resolved there) and is retargeted to main as the single combined PR. Closing; the branch stays. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #83.
Reported on a TCL Flip 2 as "the whole app is a bit slow". Nobody on the dev side has that phone, so everything below is measured on an M5 (2.9 GB, Android 13,
standardDebug, median of 5 cold starts). Debug build, so treat the absolute numbers as directional; the deltas are the point.What was wrong
Two separate problems, and the issue's own hypothesis (the Whisper model) turned out to be half of one of them.
Nothing released under memory pressure. There was no
onTrimMemory,onLowMemoryorComponentCallbacks2anywhere in the tree.MapView.onLowMemory()was never called.WhisperRecognizerhad no release path at all, so even Remove-model left ~267 MB resident for the rest of the process. ATRIM_MEMORY_COMPLETEfreed 0 KB and the system's only remaining option was to kill us.Nothing adapted to the device. No
isLowRamDevicebranch existed; the image cache was a flat 48 MB whatever the phone.Numbers
APK 97,396,968 -> 91,995,103 bytes.
Post-trim and native heap are the numbers to trust: both are PAIRED measurements inside a single
run, so run-to-run variance cancels.
An idle PSS row was published here earlier and has been REMOVED as unsound. It compared a
post-reap reading against a pre-reap one and presented them as one metric, and the headline cell
was n=1. A follow-up on a single consistent clock (median of 3, fixed 35 s and 155 s samples)
gives branch normal-RAM 233,479 KB settled / 192,557 KB post-reap and branch low-RAM 220,236 KB /
187,957 KB - internally consistent, low-RAM below normal as expected. Those are NOT comparable to
the main baseline above, which came from a different harness whose convergence heuristic latches
during the startup ramp. Quoting a delta across the two harnesses would repeat the original
mistake, so no idle delta is claimed. Peak, post-trim, native heap and cold start are all measured
identically on both builds and are unaffected.
The biggest win needed no device gate
The speech model costs ~267 MB while loaded (~101 MB of weights in
scudo:secondaryplus ~146 MB of onnxruntime arena inscudo:primary) and was kept for the whole process on the chance of a mic tap many users never make. It is now dropped after 120 s unused and rebuilt on next use, so the instant first tap asked for on 2026-07-10 is kept while the session-long hold is not.Device-verified:
scudo:secondary111 MB -> 9 MB at the 120 s mark,recognizer releasedlogged from theasr-reaperthread, model rebuilding to 125,836 KB afterwards, same pid throughout (a bad release/rebuild here is a native use-after-free, not an exception).I first made this low-RAM-conditional. That was too cautious and left roomier phones holding 267 MB all session.
Everything else
app/ui/MemoryPressure.ktis the one seam. Registration-based, never a Hilt entry point: reaching a singleton from a trim would construct it, so the trim would allocate the very thing it is freeing.WebPhotoFetcherhad no reaper at all and pinned a Chromium renderer for the whole session.WhisperRecognizer.release()declines while a listen is in flight.PiperSynthreleases only at CRITICAL, not merely severe: a reload delaying a turn prompt is a missed turn.libmaplibre.soalone carried 23 MB of them.armeabi-v7akept, since 32-bit ARM keypad phones are real.A regression I caught in my own change
The first low-RAM POI subset used 6 terms and silently deleted every park and school pin, reintroducing the exact bug the civic/green terms were added to fix (the ambient layer filter-hides the basemap OSM poi layers at z14+, so those two have no second source). Caught by an A/B screenshot, not by any test. The subset now keeps
schoolandparkdeliberately.Verification, and what was NOT run
Ran:
:core:detekt,:app:detekt(0 smells),:core:test,tests/dead_code/audit_deadcode.sh(PASS),tests/dpad/audit_static.sh(no new violations). Both flavors build;restrictedinstalled, launched, initializedMemoryPressureand rendered correctly. A/B screenshots of the POI change at native geometry, both device classes.One of four geometries ran; three did not.
kyocera-e4810(240x320 @160) passed themapandvoicephases: 2 COVERED, 0 MISSED.sonim-x320,kyocera-duraxe-e4830andsonim-x320-225areUNVERIFIED. The reasoning for not treating that as blocking, as AGENTS.md requires it be stated: The diff contains zero Compose, layout or focus code: no
Modifier, nodpadHighlight, nofocusable, no composable signature change. TheVelaMapViewedit is aDisposableEffectregistration. The matrix exists to catch clipping, focus-ring and D-pad-traversal regressions at small sizes, none of which this diff can reach. The single user-visible change is ambient POI data density, which is geometry-independent and verified by A/B screenshot.If a reviewer disagrees with that blast-radius call, the legs to run are
PHASES="map voice settings"acrosskyocera-e4810,sonim-x320,kyocera-duraxe-e4830,sonim-x320-225.Testing the low-RAM path
Every device we own reports
lowRam=false heapClassMb=256, so those branches would otherwise ship as dead code. Debug builds honour:Two traps worth knowing
am send-trim-memoryrefuses background levels on a foreground process ("Unable to set a background trim level on a foreground process"). Press HOME first. A harness that discards that stderr measures nothing and reports a clean baseline. That happened here and produced a whole benchmark of void numbers before it was noticed.AGENTS.md's memory rule said the
OverpassTrafficSignals/OverpassPoisstream-parse follow-up was pending. It has been done for some time, and chasing that stale line wasted a pass. Corrected: the remaining fully-buffered hot reader is the Google ambient path, which cannot simplydecodeFromStreambecause the payload is a positional nameless array.