SoftwareApp
1f94d0a38a
docs/PERFORMANCE.md has been honest that three of its four numbers do not exist. They exist now as measurements, if not yet as results: benchmark/ is a macrobenchmark module with four tests - cold start with StartupTimingMetric, then opening, scrolling and searching a folder of a thousand files with FrameTimingMetric - sharing a fixture at /sdcard/AABenchmark that is created once per run and deleted in @AfterClass. Frames rather than a named trace section on purpose: what a user calls slow is jank on screen, and the alternative puts benchmarking calls into production code for the sake of a number. CompilationMode.None() is the honest cold-install baseline; Partial() plus a baseline profile is the follow-up and needs the same module. The storage grant is applied with appops in setup, which is what lets the run be unattended - a permission dialog is the one thing a test cannot answer. Three build facts had to be discovered rather than designed, and each is now written where it will be read. The app under test cannot keep its own package id: it needs a debug signature to install on a test device, so it cannot replace a release install, and an un-suffixed build is exactly that. It also cannot use the obvious suffix: `.benchmark` is already the benchmark module's own APK id, and the test APK then fails to install with INSTALL_FAILED_VERSION_DOWNGRADE - which is what the first attempt did. Hence `.perf`. And `testBuildType = "benchmark"` cannot be set unconditionally, because it makes benchmark the variant that *unit* tests run against and :app:testDebugUnitTest - the task the entire gate is built on - stops existing; it is behind -PbenchmarkBuild instead. All three were measured, not guessed, and the third would have broken CI silently. Two smaller things came with it. tools/verify.sh --release now also compiles the app's benchmark variant and the test APK, because build wiring nobody runs is wiring that rots; no device is involved. And the code map grew an EXTRA_ROOTS section so it indexes benchmark/ too - a map that silently omits a module lies by omission, which is worse than the one line it costs. The numbers themselves are still missing, and for a device reason rather than a code one: installing the new package id on this MIUI phone fails with INSTALL_FAILED_USER_RESTRICTED (the "Install via USB" gate, which wants a human and answered too late). Both APKs are pushed to /sdcard/Download/ for a manual install, and the run is one command away: `adb shell am instrument -w -e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner`. Gate green with --release, including the new compile step; map current; 386 tests unchanged.
112 lines
6.4 KiB
Markdown
112 lines
6.4 KiB
Markdown
# Performance
|
||
|
||
What has been measured, on what, and — explicitly — what has **not** been measured at all. The rule
|
||
this file exists to enforce comes from the competitive survey: the market leader's top complaint is
|
||
thumbnails that load late, its own support reply to a speed complaint was *"your SD card is slow"*,
|
||
and the lesson recorded there was to measure what was measured and never close a performance
|
||
complaint by blaming the hardware.
|
||
|
||
Until this file existed, the app had **no performance number of any kind**.
|
||
|
||
## Cold start
|
||
|
||
| build | device | runs (ms) | median |
|
||
|---|---|---|---|
|
||
| debug, `1.0.0-debug` | Xiaomi phone, Android 15 | 1 758 · 1 773 · 1 814 · 1 926 · 2 043 | **1 814 ms** |
|
||
|
||
Taken with `./tools/measure-startup.sh 5`: `am start -W`, and one warm-up start discarded because the
|
||
first launch after an install pays for dexopt and folding that into a median is how a baseline gets
|
||
written that nothing can meet.
|
||
|
||
The debug build is deliberately the *slow* case — StrictMode on, LeakCanary attached, no R8 — so this
|
||
is an upper bound rather than what a user sees. **The release number is the one worth publishing, and
|
||
it has not been taken yet.** Until it is, no speed claim belongs in the README or the store listing.
|
||
|
||
### What StrictMode found on the first cold start
|
||
|
||
The same debug build now logs StrictMode violations, and the first cold start produced five, against a
|
||
1 814 ms start:
|
||
|
||
| duration | where | ours? |
|
||
|---|---|---|
|
||
| 492 ms | `BrowserViewModel.Factory` → `app.cacheDir` | **yes** — `Context.getCacheDir()` checks and creates the private cache directory while the view model is being created, on the main thread |
|
||
| 397 ms | `BrowserViewModel.rootsOf` → `StorageProvider.space` → `VolumeRepository.stat` | **yes** — free space for every volume is read with `StatFs` inside `refresh()`, which the screen calls from a `LaunchedEffect` |
|
||
| 477 ms | `FileManagerTheme` → `typography = FileManagerTypography` | no — the platform loading its own typefaces on first composition |
|
||
| 54 ms × 2 | `miui.util.font.FontSettings` via `MainActivity.onCreate` | no — the OEM's font hook doing `File.exists` while the activity is created |
|
||
|
||
That is **~1.5 s of main-thread disk I/O inside a 1.8 s start**, of which **~0.9 s is ours**. Both of
|
||
ours are mechanical to move and neither is fixed yet: the cache directory is only needed when a preview
|
||
copies something, and the free-space line on the roots screen can arrive a frame later. They are the
|
||
first items of the speed work.
|
||
|
||
## Measured on demand: the `:benchmark` module
|
||
|
||
`benchmark/` holds four macrobenchmark tests that measure the interaction numbers above on a benchmark
|
||
build of the app: release rules (R8 on, not debuggable), debug-signed, and profileable — under its own
|
||
package id, `…filemanager.perf`, so it can never collide with the test APK or take over a release
|
||
install on the device.
|
||
|
||
| benchmark | metric | what it answers |
|
||
|---|---|---|
|
||
| `coldStart` | `StartupTimingMetric` | tap → first frame |
|
||
| `openAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while a 1 000-file folder fills in |
|
||
| `scrollAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while scrolling that folder |
|
||
| `searchInsideAFolderOfAThousandFiles` | `FrameTimingMetric` | jank while typing into search |
|
||
|
||
They share a fixture — `/sdcard/AABenchmark`, a thousand empty files, created once per run and deleted
|
||
in `@AfterClass` — and the storage grant is applied with `appops`, so an unattended run is possible.
|
||
`CompilationMode.None()` is deliberate: no profile is applied, which is the honest cold-install
|
||
baseline. `CompilationMode.Partial()` plus a baseline profile is the follow-up, and needs the same
|
||
module.
|
||
|
||
**They have not produced numbers yet.** On this phone `adb install` of the new package id is refused with
|
||
`INSTALL_FAILED_USER_RESTRICTED` — MIUI's "Install via USB" gate, which wants a human — so the two APKs
|
||
were pushed to `/sdcard/Download/` (`PlainFiles-perf.apk`, `PlainFiles-benchmark-test.apk`) to be
|
||
installed by hand. Once they are on the device:
|
||
|
||
```bash
|
||
adb shell am instrument -w \
|
||
-e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark \
|
||
de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner
|
||
```
|
||
|
||
With a device that accepts installs, the Gradle path does the same and writes the numbers next to the
|
||
build output:
|
||
|
||
```bash
|
||
./gradlew -PbenchmarkBuild :benchmark:connectedDebugAndroidTest
|
||
```
|
||
|
||
`-PbenchmarkBuild` is not decoration: it is what points the app's instrumentation at the `benchmark`
|
||
build type. Setting that unconditionally in the build file makes `benchmark` the variant *unit* tests
|
||
run against, and `:app:testDebugUnitTest` — the task the gate is built on — disappears. Measured, not
|
||
guessed.
|
||
|
||
## Not measured yet, and why
|
||
|
||
| number | how it would be taken | why it is not here |
|
||
|---|---|---|
|
||
| Folder open, 1 000 / 5 000 / 20 000 files | macrobenchmark `StartupTimingMetric` over a fixture volume, or `TraceSectionMetric` around the listing | needs a `:benchmark` module and a profileable build installed under the **release package id**; on this MIUI phone the first install of a new package id waits for a confirmation dialog, so it cannot run unattended |
|
||
| Thumbnail paint while scrolling a photo grid | `FrameTimingMetric` | same |
|
||
| Keystroke → search result | `TraceSectionMetric` around the query and the index read | same |
|
||
|
||
The module does not exist yet, deliberately: half-configured build wiring that nobody can run is worse
|
||
than an honest gap. It is the next measurement step, and it belongs with the baseline-profile work,
|
||
which needs the same module.
|
||
|
||
## What is measured instead, meanwhile
|
||
|
||
The gate measures things that are not latency but do not rot: the test count, lint, detekt against its
|
||
baseline, and whether `docs/CODE_MAP.md` still matches the tree. Those are the numbers that can be
|
||
kept true by automation; the three above need a device and a decision about which device.
|
||
|
||
## Two rules
|
||
|
||
- **No number that was not taken from a real build on a real device.** Not from a debug build
|
||
pretending to be release, and not from reading the code.
|
||
- **No performance change judged by reading the code.** The bigger-thumbnail change (28 → 48 dp in a
|
||
row, 40 → 96 dp in a grid) is the standing example: it may be exactly right, and nobody measured
|
||
either side of it. `docs/COMPETITIVE_ANALYSIS.md` had already written "a thumbnail that is bigger and
|
||
slower is strictly worse than one that is smaller and instant" — which is a hypothesis until it is a
|
||
number.
|