fishfacegit/file-manager
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Record the first complete benchmark run, and what its numbers are not

Browse Source 
With the install watcher answering MIUI's dialog, all four tests ran and three produced numbers: coldStart 574.17 ms P50 (553.95-730.19), opening a 1000-file folder 14 frames P50 (13-17), scrolling it with four swipes 98 frames P50 (93-100). The search benchmark still fails, on the app bar's search action not being in the accessibility tree after twenty seconds - the setup wait is raised from ten to twenty now that the folder render is known to be the slow step.

The frame counts are labelled as frame counts and not as jank, because this phone reports no frame durations at all: the same results file contains frameCount and nothing else, so 14 frames says the interaction is short and says nothing about whether a frame was slow. The doc also keeps the measurements that explain the watcher's shape - the dialog is FLAG_SECURE, its lifetime varies between under a second and 8.5 s, and pm install from an app uid fails earlier with MIUI's XSpace SecurityException - along with the honest cost: MIUI keeps mCurrentFocus on the installer after the window is gone, so a coordinate tap can land on the home screen, which it did. The watcher no longer reuses a saved position at all and refuses to tap within two seconds of the previous tap.
This commit is contained in:
Hadrian Burkhardt
2026-10-17 15:26:59 +02:00
parent 456884a4d0
commit b72847b822
4 changed files with 72 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CODE_MAP.md
+2 -2
View File
@@ -2,7 +2,7 @@
Generated from the source by `tools/make-code-map.py` - do not edit by hand. Regenerate it after a change (`python3 tools/make-code-map.py`); `--check` exits non-zero when this file is stale.
77 source files, 12704 lines - 33 test files, 386 tests. The behaviour index at the end lists every test name, and in this project a test name is a sentence about what the app does, so it doubles as the specification.
77 source files, 12707 lines - 33 test files, 386 tests. The behaviour index at the end lists every test name, and in this project a test name is a sentence about what the app does, so it doubles as the specification.
Start at `AGENTS.md`. What is done, and why: `docs/ROADMAP.md` and `CHANGELOG.md`. What was verified by hand rather than by a test: `docs/EDGE_CASE_TESTING.md`.
@@ -170,7 +170,7 @@ The macrobenchmark test APK: four measurements over a fixture folder, run on a d
| file | lines | what it holds | tests |
|---|---|---|---|
| `BrowserBenchmark.kt` | 212 | The four numbers docs/PERFORMANCE.md promises, measured on a benchmark build of the app. BrowserBenchmark, Fixture | - |
| `BrowserBenchmark.kt` | 215 | The four numbers docs/PERFORMANCE.md promises, measured on a benchmark build of the app. BrowserBenchmark, Fixture | - |
## Behaviour index
docs/PERFORMANCE.md
+58 -45
View File
@@ -18,10 +18,10 @@ Until this file existed, the app had **no performance number of any kind**.
The first is `./tools/measure-startup.sh 5` (`am start -W`, one warm-up start discarded because the first
launch after an install pays for dexopt). The second is `BrowserBenchmark#coldStart` in the `:benchmark`
module: `StartupTimingMetric`, five iterations, `CompilationMode.None()`, measured as time to initial
display — min 499 ms, median 532 ms, max 805 ms. Measured three times: 615 ms before the storage grant was
in place (the app was showing the permission screen), 532 ms after it, and 560 ms (min 539, max 698) on a
later run that also exercised the full benchmark suite. The three runs agree within a phone's patience;
the 532 ms figure above is the middle one.
display — min 499 ms, median 532 ms, max 805 ms. Measured four times: 615 ms before the storage grant was
in place (the app was showing the permission screen), 532 ms after it, 560 ms (min 539, max 698) on an
earlier full-suite run, and 574 ms (min 554, max 730) on the run that first completed all four benchmarks.
The four runs agree within a phone's patience; the 532 ms figure above is the middle one.
Two caveats belong to that number rather than to a footnote. The tool itself warns *"Unable to clear
Runtime Image, subsequent launches/iterations may exhibit faster startup than production due to
@@ -74,46 +74,58 @@ and removed with `--remove`. `CompilationMode.None()` is deliberate: no profile
honest cold-install baseline. `CompilationMode.Partial()` plus a baseline profile is the follow-up, and
needs the same module.
### What the module has actually measured
### What the module has measured
| benchmark | outcome on the Xiaomi phone (Android 12, MIUI) |
|---|---|
| `coldStart` | **passes** — 560.6 ms P50, min 539, max 698 over five iterations. A second measurement of the 532 ms above, and the only number this toolchain reports here |
| `scrollAFolderOfAThousandFiles` | passes, but reports **only `frameCount`** (46–100 frames per iteration): this device hands macrobenchmark no `frameDurationCpuMs`/`frameOverrunMs` at all |
| `openAFolderOfAThousandFiles` | `INSTALL_FAILED_USER_RESTRICTED: Install canceled by user` — the rule reinstalls the app before measuring, and MIUI refuses that reinstall |
| `searchInsideAFolderOfAThousandFiles` | the app bar's search action is not in the accessibility tree within ten seconds of the folder opening, though it is there when the same taps are made by hand |
The run that first completed all four tests (MIUI's install dialog answered by
`tools/benchmark-watch-install.sh`, 5 iterations each, `CompilationMode.None()`):
So the interaction numbers are not written down here because **the device will not produce them**, not
because the code cannot ask. Two separate device behaviours are in the way, and both are outside the app:
a reinstall MIUI refuses, and a platform that reports frame counts but not frame durations.
| benchmark | result | what it is |
|---|---|---|
| `coldStart` | **574.17 ms** P50 (min 553.95, max 730.19) | time to initial display — the fourth measurement of the same thing, alongside 615 / 532 / 560 ms |
| `openAFolderOfAThousandFiles` | **14 frames** P50 (min 13, max 17) | frames drawn from the folder tap until `file-0.txt` is on screen |
| `scrollAFolderOfAThousandFiles` | **98 frames** P50 (min 93, max 100) | frames drawn across four swipes through the folder |
| `searchInsideAFolderOfAThousandFiles` | **failed** | the app bar's search action is not in the accessibility tree within 20 s of the folder opening, although the same taps by hand find it immediately |
### Why the reinstall cannot be answered
**Those frame counts are not a jank metric, and the difference matters.** `FrameTimingMetric` normally
publishes `frameDurationCpuMs` and `frameOverrunMs` — how long each frame took and how far past its
deadline it ran. This phone reports neither: the same run's JSON contains `frameCount` and nothing else,
so "14 frames to open the folder" is how many frames the interaction took, not how long any of them
lasted. It says the interaction is short (14 frames is about a quarter of a second at 60 Hz); it cannot
say whether any of those frames was slow. A device that reports frame durations — an emulator or any
non-MIUI phone — is what turns this into a jank number.
`tools/benchmark-watch-install.sh` exists to answer MIUI's confirmation from the host, side by side with a
run: it polls every five seconds, reads the screen when an installer window has focus, and taps the
**Install** button's own coordinates — never a guessed position. It works for an install that stays up
(`adb install` of a new package: the dialog appears after 1.7 s, counts down from ten, and the script
reads the button and taps it), and it does **not** work for the one that matters. Measured:
Two notes on provenance. The results JSON from that run was deleted by the next run's cleanup before it
could be copied a second time, so the figures above come from parsing it once, immediately after the run;
re-running on a device without an install gate is the way to confirm them. And the run still ends with a
failing `search` test, so its number is missing rather than zero.
### Why the reinstall cannot be answered by the app
macrobenchmark installs its target before measuring it, and on this phone MIUI puts that install behind
`com.miui.securitycenter/.permcenter.install.AdbInstallActivity`. The test cannot answer it itself:
`CompilationMode.installPackageFromPaths` builds `pm install -t …` and runs it through
`Shell.executeScriptCaptureStdout`, so the call blocks until the child exits and no test code is running
while the dialog is up.
`tools/benchmark-watch-install.sh` answers it from the host instead — but it is a workaround with a cost,
and it is not part of the benchmark workflow for that reason: MIUI keeps `mCurrentFocus` pointing at the
installer for a moment after the window has been removed, so a tap sent on that signal can land on the
home screen underneath (observed: it opened Telegram). The script now reads the button off the screen
before every tap and refuses to tap when the installer is not focused, which narrows the race to the
handful of milliseconds between that check and the tap — it does not close it. MIUI's own **Install via
USB** setting is what removes the dialog; with it on, installs need no tap at all.
Three further measurements about that dialog, kept because they explain the workaround's shape:
- The dialog the rule's install raises is focused for **well under a second** —
`13:51:27.70 AdbInstallActivity` → `13:51:28.38 null` in a 0.1 s focus trace, and one sample in a 0.5 s
trace. One `uiautomator dump` costs 3–4 s, so the button cannot even be read before the window is gone.
- It is `FLAG_SECURE`: a `screencap` taken while `mCurrentFocus` was the installer shows the home screen
behind it, which is why screenshots of "the dialog" show nothing.
- A tap on it, when a run did stay up long enough for the script to read and tap `Installieren`, still
ended with `INSTALL_FAILED_USER_RESTRICTED: Install canceled by user`. The same happened later with the
script answering *two* dialogs in one run from a measured button position, each logged with the
installer window still focused: the install was refused anyway. MIUI is closing its own dialog when the
install session ends, so the tap arrives after the outcome has been decided.
- Its lifetime is not stable. One 0.1 s focus trace caught it as
`13:51:27.70 AdbInstallActivity` → `13:51:28.38 null` — too short to read, let alone tap — while a later
trace showed the same window focused for 8.5 s, long enough for a dump and a tap. Presumably the
difference is whether MIUI is waiting for an answer or merely announcing the install.
- `pm install` started from an *app* uid does not even reach the gate: MIUI's `XSpaceManagerServiceImpl`
throws `SecurityException: You either need MANAGE_USERS or CREATE_USERS permission to: query users`.
So the rule's install runs through the shell, MIUI intercepts it as an "Install via USB" request, and
gates *new* packages while letting updates through — which is why `adb install -r` of an
already-installed build returns `Success` instantly and the rule's reinstall never does.
The switch that governs this is MIUI's own **Developer options → Install via USB**, whose name the dialog
carries. It is not in any Android settings namespace, so it cannot be read or set away from the phone's
own UI.
The rule's install therefore runs through the shell, which is the path MIUI intercepts.
### The four switches that make an unattended run possible at all
@@ -155,17 +167,18 @@ what recovered the numbers above.
## Not measured yet, and why
| number | how it would be taken | why it is not here |
|---|---|---|
| Folder open, 1 000 / 5 000 / 20 000 files | `FrameTimingMetric` around the folder tap (`openAFolderOfAThousandFiles` exists) | the rule reinstalls the app mid-run, and MIUI answers that install with **"Install canceled by user"** unless someone is there to tap |
| Thumbnail paint while scrolling a photo grid | `FrameTimingMetric` | same; and the scroll variant that *did* run reported only `frameCount`, because this device gives macrobenchmark no frame durations |
| Keystroke → search result | `FrameTimingMetric` around typing into search (`searchInsideAFolderOfAThousandFiles` exists) | the app bar's search action does not appear in the accessibility tree within ten seconds of the folder opening in the automated run, although the same taps by hand find it immediately |
| number | status |
|---|---|
| Folder open with 1 000 files | **14 frames** P50 — measured, but as a frame *count*, not a jank figure |
| Folder open with 5 000 / 20 000 files | not measured: the fixture script counts to 1 000, and every larger run needs another install cycle |
| Scrolling that folder | **98 frames** P50 over four swipes — again a frame count |
| Typing into search | **failed** in the automated run: the app bar's search action is not in the accessibility tree within 20 s of the folder opening, although the same taps by hand find it immediately |
| Thumbnail paint while scrolling a photo grid | not measured: no test for it yet, and no frame durations on this device |
The module exists and three of its four tests pass; what is missing is a device that lets macrobenchmark
install unattended *and* reports frame timing. A plain emulator image (or any non-MIUI phone) with the
system language set to English would answer all three — the fixture is a folder of empty files, so nothing
device-specific is being measured. Until then this file says what was measured and what was not, and the
cold-start number stands alone.
What the interaction numbers need is a device that reports frame durations; what the *repeated* runs need
is a device that installs unattended. This phone gives neither, and the two are independent: a
non-MIUI phone or an emulator with the system language set to English would answer both, and the fixture
is a folder of empty files, so nothing device-specific is being measured.
## What is measured instead, meanwhile