fishfacegit/file-manager
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Get a real cold-start number out of the benchmark, and record what the device demanded

Browse Source 
coldStart measures 532 ms median (min 499, max 805) on the benchmark build - R8, not debuggable, profileable, CompilationMode.None - against 1814 ms on the debug build, which is the honest illustration of why a debug number must never be published as the app's performance. The tool also warns that it cannot clear the runtime image on this Android version, so the figure is optimistic by an unmeasured amount; that is in PERFORMANCE.md next to the number rather than in a footnote.

The three interaction benchmarks are written and compile but have not produced a number, and getting that far took four fixes worth naming. The app under test cannot keep the app's package id (signature) and cannot use .benchmark (the test APK owns that id, INSTALL_FAILED_VERSION_DOWNGRADE); macrobenchmark needs package visibility for it or it reports NameNotFoundException for an app that pm list packages shows installed; UiAutomation.executeShellCommand runs as the test app's uid, so the fixture has to be made through adb instead - that one cost the first attempt a hang; and testBuildType has to be behind a flag or :app:testDebugUnitTest stops existing.

The workflow that works on a device with an install gate is now documented: install the two APKs by hand once, run tools/benchmark-fixture.sh, then am instrument. The Gradle task is documented as the alternative and as the reason the app under test disappeared once - it uninstalls around the run.

Gate green, map current, 386 tests unchanged.
This commit is contained in:
Hadrian Burkhardt
2026-10-17 10:15:28 +02:00
parent 1f94d0a38a
commit 94f85bd287
5 changed files with 112 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PERFORMANCE.md
+33 -21
View File
@@ -13,14 +13,23 @@ Until this file existed, the app had **no performance number of any kind**.
| build | device | runs (ms) | median |
|---|---|---|---|
| debug, `1.0.0-debug` | Xiaomi phone, Android 15 | 1 758 · 1 773 · 1 814 · 1 926 · 2 043 | **1 814 ms** |
| `benchmark` (R8, not debuggable, profileable) | same | — | **532 ms** |
Taken with `./tools/measure-startup.sh 5`: `am start -W`, and one warm-up start discarded because the
first launch after an install pays for dexopt and folding that into a median is how a baseline gets
written that nothing can meet.
The first is `./tools/measure-startup.sh 5` (`am start -W`, one warm-up start discarded because the first
launch after an install pays for dexopt). The second is `BrowserBenchmark#coldStart` in the `:benchmark`
module: `StartupTimingMetric`, five iterations, `CompilationMode.None()`, measured as time to initial
display — min 499 ms, median 532 ms, max 805 ms. Measured twice: 615 ms before the storage grant was in
place (the app was showing the permission screen) and 532 ms after, which is the figure above.
The debug build is deliberately the *slow* case — StrictMode on, LeakCanary attached, no R8 — so this
is an upper bound rather than what a user sees. **The release number is the one worth publishing, and
it has not been taken yet.** Until it is, no speed claim belongs in the README or the store listing.
Two caveats belong to that number rather than to a footnote. The tool itself warns *"Unable to clear
Runtime Image, subsequent launches/iterations may exhibit faster startup than production due to
accelerated class loading"*, so it is optimistic by an unmeasured amount on this Android version. And
`CompilationMode.None()` means no compilation profile is applied — what a fresh install does — so a
baseline profile would lower it further and is the obvious next measurement.
The debug build is deliberately the *slow* case — StrictMode on, LeakCanary attached, no R8 — which is
why the same screen takes it three times as long. That is the honest illustration of why a debug number
must never be published as the app's performance.
### What StrictMode found on the first cold start
@@ -59,28 +68,31 @@ in `@AfterClass` — and the storage grant is applied with `appops`, so an unatt
baseline. `CompilationMode.Partial()` plus a baseline profile is the follow-up, and needs the same
module.
**They have not produced numbers yet.** On this phone `adb install` of the new package id is refused with
`INSTALL_FAILED_USER_RESTRICTED` — MIUI's "Install via USB" gate, which wants a human — so the two APKs
were pushed to `/sdcard/Download/` (`PlainFiles-perf.apk`, `PlainFiles-benchmark-test.apk`) to be
installed by hand. Once they are on the device:
**Two of these have produced a number** (`coldStart`, above); the three interaction ones have not, and
the reason is the device rather than the code. Getting them to run at all took four fixes, each of which
is written next to the code it belongs to: the app under test needs its own package id; it needs to be
installed by hand once, because MIUI refuses `adb install` of a new package id without a human
(`INSTALL_FAILED_USER_RESTRICTED`); the test APK needs package-visibility for the app it measures, or
macrobenchmark reports `NameNotFoundException` for an app that `pm list packages` shows installed; and
the fixture has to be created through adb, because `UiAutomation.executeShellCommand` runs with the
*test app's* uid and cannot write to `/sdcard`.
The workflow that avoids all of it after the first install:
```bash
# once, by hand on the phone (Files → Download → tap), because MIUI gates adb installs:
# /sdcard/Download/PlainFiles-perf.apk (app under test, package …filemanager.perf)
# /sdcard/Download/PlainFiles-benchmark-test.apk
./tools/benchmark-fixture.sh # create the fixture and grant storage access
adb shell am instrument -w \
-e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark \
de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner
```
With a device that accepts installs, the Gradle path does the same and writes the numbers next to the
build output:
```bash
./gradlew -PbenchmarkBuild :benchmark:connectedDebugAndroidTest
```
`-PbenchmarkBuild` is not decoration: it is what points the app's instrumentation at the `benchmark`
build type. Setting that unconditionally in the build file makes `benchmark` the variant *unit* tests
run against, and `:app:testDebugUnitTest` — the task the gate is built on — disappears. Measured, not
guessed.
`am instrument` is preferred over `./gradlew -PbenchmarkBuild :benchmark:connectedDebugAndroidTest` on a
device with an install gate: the Gradle task installs *and uninstalls* the APKs around the run, so every
run needs a human again — and an uninstall between runs is what made the app under test disappear once.
## Not measured yet, and why