94f85bd287
coldStart measures 532 ms median (min 499, max 805) on the benchmark build - R8, not debuggable, profileable, CompilationMode.None - against 1814 ms on the debug build, which is the honest illustration of why a debug number must never be published as the app's performance. The tool also warns that it cannot clear the runtime image on this Android version, so the figure is optimistic by an unmeasured amount; that is in PERFORMANCE.md next to the number rather than in a footnote. The three interaction benchmarks are written and compile but have not produced a number, and getting that far took four fixes worth naming. The app under test cannot keep the app's package id (signature) and cannot use .benchmark (the test APK owns that id, INSTALL_FAILED_VERSION_DOWNGRADE); macrobenchmark needs package visibility for it or it reports NameNotFoundException for an app that pm list packages shows installed; UiAutomation.executeShellCommand runs as the test app's uid, so the fixture has to be made through adb instead - that one cost the first attempt a hang; and testBuildType has to be behind a flag or :app:testDebugUnitTest stops existing. The workflow that works on a device with an install gate is now documented: install the two APKs by hand once, run tools/benchmark-fixture.sh, then am instrument. The Gradle task is documented as the alternative and as the reason the app under test disappeared once - it uninstalls around the run. Gate green, map current, 386 tests unchanged.
124 lines
7.8 KiB
Markdown
124 lines
7.8 KiB
Markdown
# Performance
|
||
|
||
What has been measured, on what, and — explicitly — what has **not** been measured at all. The rule
|
||
this file exists to enforce comes from the competitive survey: the market leader's top complaint is
|
||
thumbnails that load late, its own support reply to a speed complaint was *"your SD card is slow"*,
|
||
and the lesson recorded there was to measure what was measured and never close a performance
|
||
complaint by blaming the hardware.
|
||
|
||
Until this file existed, the app had **no performance number of any kind**.
|
||
|
||
## Cold start
|
||
|
||
| build | device | runs (ms) | median |
|
||
|---|---|---|---|
|
||
| debug, `1.0.0-debug` | Xiaomi phone, Android 15 | 1 758 · 1 773 · 1 814 · 1 926 · 2 043 | **1 814 ms** |
|
||
| `benchmark` (R8, not debuggable, profileable) | same | — | **532 ms** |
|
||
|
||
The first is `./tools/measure-startup.sh 5` (`am start -W`, one warm-up start discarded because the first
|
||
launch after an install pays for dexopt). The second is `BrowserBenchmark#coldStart` in the `:benchmark`
|
||
module: `StartupTimingMetric`, five iterations, `CompilationMode.None()`, measured as time to initial
|
||
display — min 499 ms, median 532 ms, max 805 ms. Measured twice: 615 ms before the storage grant was in
|
||
place (the app was showing the permission screen) and 532 ms after, which is the figure above.
|
||
|
||
Two caveats belong to that number rather than to a footnote. The tool itself warns *"Unable to clear
|
||
Runtime Image, subsequent launches/iterations may exhibit faster startup than production due to
|
||
accelerated class loading"*, so it is optimistic by an unmeasured amount on this Android version. And
|
||
`CompilationMode.None()` means no compilation profile is applied — what a fresh install does — so a
|
||
baseline profile would lower it further and is the obvious next measurement.
|
||
|
||
The debug build is deliberately the *slow* case — StrictMode on, LeakCanary attached, no R8 — which is
|
||
why the same screen takes it three times as long. That is the honest illustration of why a debug number
|
||
must never be published as the app's performance.
|
||
|
||
### What StrictMode found on the first cold start
|
||
|
||
The same debug build now logs StrictMode violations, and the first cold start produced five, against a
|
||
1 814 ms start:
|
||
|
||
| duration | where | ours? |
|
||
|---|---|---|
|
||
| 492 ms | `BrowserViewModel.Factory` → `app.cacheDir` | **yes** — `Context.getCacheDir()` checks and creates the private cache directory while the view model is being created, on the main thread |
|
||
| 397 ms | `BrowserViewModel.rootsOf` → `StorageProvider.space` → `VolumeRepository.stat` | **yes** — free space for every volume is read with `StatFs` inside `refresh()`, which the screen calls from a `LaunchedEffect` |
|
||
| 477 ms | `FileManagerTheme` → `typography = FileManagerTypography` | no — the platform loading its own typefaces on first composition |
|
||
| 54 ms × 2 | `miui.util.font.FontSettings` via `MainActivity.onCreate` | no — the OEM's font hook doing `File.exists` while the activity is created |
|
||
|
||
That is **~1.5 s of main-thread disk I/O inside a 1.8 s start**, of which **~0.9 s is ours**. Both of
|
||
ours are mechanical to move and neither is fixed yet: the cache directory is only needed when a preview
|
||
copies something, and the free-space line on the roots screen can arrive a frame later. They are the
|
||
first items of the speed work.
|
||
|
||
## Measured on demand: the `:benchmark` module
|
||
|
||
`benchmark/` holds four macrobenchmark tests that measure the interaction numbers above on a benchmark
|
||
build of the app: release rules (R8 on, not debuggable), debug-signed, and profileable — under its own
|
||
package id, `…filemanager.perf`, so it can never collide with the test APK or take over a release
|
||
install on the device.
|
||
|
||
| benchmark | metric | what it answers |
|
||
|---|---|---|
|
||
| `coldStart` | `StartupTimingMetric` | tap → first frame |
|
||
| `openAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while a 1 000-file folder fills in |
|
||
| `scrollAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while scrolling that folder |
|
||
| `searchInsideAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while typing into search |
|
||
|
||
They share a fixture — `/sdcard/AABenchmark`, a thousand empty files, created once per run and deleted
|
||
in `@AfterClass` — and the storage grant is applied with `appops`, so an unattended run is possible.
|
||
`CompilationMode.None()` is deliberate: no profile is applied, which is the honest cold-install
|
||
baseline. `CompilationMode.Partial()` plus a baseline profile is the follow-up, and needs the same
|
||
module.
|
||
|
||
**Two of these have produced a number** (`coldStart`, above); the three interaction ones have not, and
|
||
the reason is the device rather than the code. Getting them to run at all took four fixes, each of which
|
||
is written next to the code it belongs to: the app under test needs its own package id; it needs to be
|
||
installed by hand once, because MIUI refuses `adb install` of a new package id without a human
|
||
(`INSTALL_FAILED_USER_RESTRICTED`); the test APK needs package-visibility for the app it measures, or
|
||
macrobenchmark reports `NameNotFoundException` for an app that `pm list packages` shows installed; and
|
||
the fixture has to be created through adb, because `UiAutomation.executeShellCommand` runs with the
|
||
*test app's* uid and cannot write to `/sdcard`.
|
||
|
||
The workflow that avoids all of it after the first install:
|
||
|
||
```bash
|
||
# once, by hand on the phone (Files → Download → tap), because MIUI gates adb installs:
|
||
# /sdcard/Download/PlainFiles-perf.apk (app under test, package …filemanager.perf)
|
||
# /sdcard/Download/PlainFiles-benchmark-test.apk
|
||
|
||
./tools/benchmark-fixture.sh # create the fixture and grant storage access
|
||
adb shell am instrument -w \
|
||
-e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark \
|
||
de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner
|
||
```
|
||
|
||
`am instrument` is preferred over `./gradlew -PbenchmarkBuild :benchmark:connectedDebugAndroidTest` on a
|
||
device with an install gate: the Gradle task installs *and uninstalls* the APKs around the run, so every
|
||
run needs a human again — and an uninstall between runs is what made the app under test disappear once.
|
||
|
||
## Not measured yet, and why
|
||
|
||
| number | how it would be taken | why it is not here |
|
||
|---|---|---|
|
||
| Folder open, 1 000 / 5 000 / 20 000 files | macrobenchmark `StartupTimingMetric` over a fixture volume, or `TraceSectionMetric` around the listing | needs a `:benchmark` module and a profileable build installed under the **release package id**; on this MIUI phone the first install of a new package id waits for a confirmation dialog, so it cannot run unattended |
|
||
| Thumbnail paint while scrolling a photo grid | `FrameTimingMetric` | same |
|
||
| Keystroke → search result | `TraceSectionMetric` around the query and the index read | same |
|
||
|
||
The module does not exist yet, deliberately: half-configured build wiring that nobody can run is worse
|
||
than an honest gap. It is the next measurement step, and it belongs with the baseline-profile work,
|
||
which needs the same module.
|
||
|
||
## What is measured instead, meanwhile
|
||
|
||
The gate measures things that are not latency but do not rot: the test count, lint, detekt against its
|
||
baseline, and whether `docs/CODE_MAP.md` still matches the tree. Those are the numbers that can be
|
||
kept true by automation; the three above need a device and a decision about which device.
|
||
|
||
## Two rules
|
||
|
||
- **No number that was not taken from a real build on a real device.** Not from a debug build
|
||
pretending to be release, and not from reading the code.
|
||
- **No performance change judged by reading the code.** The bigger-thumbnail change (28 → 48 dp in a
|
||
row, 40 → 96 dp in a grid) is the standing example: it may be exactly right, and nobody measured
|
||
either side of it. `docs/COMPETITIVE_ANALYSIS.md` had already written "a thumbnail that is bigger and
|
||
slower is strictly worse than one that is smaller and instant" — which is a hypothesis until it is a
|
||
number.
|