Stop macrobenchmark from installing at all, and measure the app that is there

The MIUI dialog was never the problem to solve; the install was. CompilationMode.None() reports shouldReset = true, which is what makes resetAndCompile uninstall the app and run pm install -t --skip-verification before every test - the one command MIUI intercepts with a FLAG_SECURE dialog that nothing inside a blocking install call can answer. CompilationMode.Ignore() reports shouldReset = false: the library installs nothing and measures what is installed. So the APK is installed once from the host and the run contains no install at all.

Measured on the phone: with Ignore() the four tests ran with zero installs - the watcher script logged heartbeats and no dialogs - and three passed. The numbers are the best yet and taken on a run with no install churn: coldStart 519.5 ms median (503.6-544.7), opening a 1000-file folder 14 frames median (11-17), scrolling it with four swipes 94 frames median (48-96). The search benchmark still fails on the app bar's search action not being in the accessibility tree.

Also recorded: the storage grant now survives because nothing uninstalls, MIUI refuses to let its SecurityCenter be disabled (Cannot disable miui core packages), and the fix for the whole afternoon is five lines in the benchmark's KDoc plus a documented am instrument command, because Gradle's own install steps are gated too.

benchmark/src/main/java/de/softwareapp_hb/filemanager/benchmark/BrowserBenchmark.kt

@@ -1,6 +1,7 @@
 package de.softwareapp_hb.filemanager.benchmark
 
 import androidx.benchmark.macro.CompilationMode
+import androidx.benchmark.macro.ExperimentalMacrobenchmarkApi
 import androidx.benchmark.macro.FrameTimingMetric
 import androidx.benchmark.macro.MacrobenchmarkScope
 import androidx.benchmark.macro.StartupMode
@@ -22,10 +23,24 @@ import org.junit.runner.RunWith
  *
  * Three decisions are worth knowing before reading the output.
  *
- * **`CompilationMode.None()`**: no compilation profile is applied, so these are the numbers of code
+ * **`CompilationMode.Ignore()`, not `None()`.** Both apply no compilation profile, but `None()` reports
- * that has never run. That is the honest baseline for "is this fast on a cold install", and it is
+ * `shouldReset = true`, and that is what makes macrobenchmark uninstall the app and reinstall it with
- * also the mode that does not need a profile file to exist. `CompilationMode.Partial()` — what a
+ * `pm install -t --skip-verification` before *every* test. On this MIUI phone that install is
- * baseline profile would buy — is the follow-up, and it needs the profile generated first.
+ * intercepted by `com.miui.securitycenter/.permcenter.install.AdbInstallActivity`, which is
+ * `FLAG_SECURE`, lives a variable amount of time, and denies the install when nobody answers — and a
+ * denied install leaves the app uninstalled, so one refusal made every later test fail with
+ * `NameNotFoundException`. `Ignore()` reports `shouldReset = false`: the library installs nothing and
+ * measures what is there. So the APK is installed once, from the host:
+ *
+ *     adb install -r app/build/outputs/apk/benchmark/app-benchmark.apk
+ *     adb shell cmd package compile --reset de.softwareapp_hb.filemanager   # the "never run" baseline
+ *     ./gradlew :benchmark:connectedBenchmarkAndroidTest
+ *
+ * The `compile --reset` is what preserves the meaning of the old `None()` numbers: no profile, so the
+ * code being measured has never run. What changes is only *who* installs the APK — and it has to be the
+ * host, because an install started inside the test cannot be answered while it blocks.
+ * `CompilationMode.Partial()` — what a baseline profile would buy — is the follow-up, and it needs the
+ * profile generated first.
  *
  * **Frames, not milliseconds.** Three of the four use [FrameTimingMetric], because what a user calls
  * slow is jank on screen, not a state transition finishing. The alternative is a named trace section
@@ -36,6 +51,7 @@ import org.junit.runner.RunWith
  * "Internal storage", "Search this folder". A localised device fails with a message saying so rather
  * than silently measuring something else.
  */
+@OptIn(ExperimentalMacrobenchmarkApi::class)
 @RunWith(AndroidJUnit4::class)
 class BrowserBenchmark {
 
@@ -89,7 +105,7 @@ class BrowserBenchmark {
     fun coldStart() = benchmark.measureRepeated(
         packageName = app,
         metrics = listOf(StartupTimingMetric()),
-        compilationMode = CompilationMode.None(),
+        compilationMode = CompilationMode.Ignore(),
         iterations = 5,
         startupMode = StartupMode.COLD,
     ) {
@@ -101,7 +117,7 @@ class BrowserBenchmark {
     fun openAFolderOfAThousandFiles() = benchmark.measureRepeated(
         packageName = app,
         metrics = listOf(FrameTimingMetric()),
-        compilationMode = CompilationMode.None(),
+        compilationMode = CompilationMode.Ignore(),
         iterations = 5,
         setupBlock = { openFixture(device) },
     ) {
@@ -113,7 +129,7 @@ class BrowserBenchmark {
     fun scrollAFolderOfAThousandFiles() = benchmark.measureRepeated(
         packageName = app,
         metrics = listOf(FrameTimingMetric()),
-        compilationMode = CompilationMode.None(),
+        compilationMode = CompilationMode.Ignore(),
         iterations = 5,
         setupBlock = {
             openFixture(device)
@@ -136,7 +152,7 @@ class BrowserBenchmark {
     fun searchInsideAFolderOfAThousandFiles() = benchmark.measureRepeated(
         packageName = app,
         metrics = listOf(FrameTimingMetric()),
-        compilationMode = CompilationMode.None(),
+        compilationMode = CompilationMode.Ignore(),
         iterations = 5,
         setupBlock = {
             openFixture(device)

docs/CODE_MAP.md

@@ -2,7 +2,7 @@
 
 Generated from the source by `tools/make-code-map.py` - do not edit by hand. Regenerate it after a change (`python3 tools/make-code-map.py`); `--check` exits non-zero when this file is stale.
 
-77 source files, 12707 lines - 33 test files, 386 tests. The behaviour index at the end lists every test name, and in this project a test name is a sentence about what the app does, so it doubles as the specification.
+77 source files, 12723 lines - 33 test files, 386 tests. The behaviour index at the end lists every test name, and in this project a test name is a sentence about what the app does, so it doubles as the specification.
 
 Start at `AGENTS.md`. What is done, and why: `docs/ROADMAP.md` and `CHANGELOG.md`. What was verified by hand rather than by a test: `docs/EDGE_CASE_TESTING.md`.
 
@@ -170,7 +170,7 @@ The macrobenchmark test APK: four measurements over a fixture folder, run on a d
 
 | file | lines | what it holds | tests |
 |---|---|---|---|
-| `BrowserBenchmark.kt` | 215 | The four numbers docs/PERFORMANCE.md promises, measured on a benchmark build of the app. BrowserBenchmark, Fixture | - |
+| `BrowserBenchmark.kt` | 231 | The four numbers docs/PERFORMANCE.md promises, measured on a benchmark build of the app. BrowserBenchmark, Fixture | - |
 
 ## Behaviour index
 

docs/PERFORMANCE.md

@@ -18,10 +18,11 @@ Until this file existed, the app had **no performance number of any kind**.
 The first is `./tools/measure-startup.sh 5` (`am start -W`, one warm-up start discarded because the first
 launch after an install pays for dexopt). The second is `BrowserBenchmark#coldStart` in the `:benchmark`
 module: `StartupTimingMetric`, five iterations, `CompilationMode.None()`, measured as time to initial
-display — min 499 ms, median 532 ms, max 805 ms. Measured four times: 615 ms before the storage grant was
+display — min 499 ms, median 532 ms, max 805 ms. Measured five times, on four different runs: 615 ms
-in place (the app was showing the permission screen), 532 ms after it, 560 ms (min 539, max 698) on an
+before the storage grant was in place (the app was showing the permission screen), 532 ms after it, 560 ms
-earlier full-suite run, and 574 ms (min 554, max 730) on the run that first completed all four benchmarks.
+(min 539, max 698), 574 ms (min 554, max 730), and 519.5 ms (min 503.6, max 544.7) on the first run that
-The four runs agree within a phone's patience; the 532 ms figure above is the middle one.
+used `CompilationMode.Ignore()`. They agree within a phone's patience; the 532 ms figure above is the middle
+one, and the later runs are the ones whose method is worth copying.
 
 Two caveats belong to that number rather than to a footnote. The tool itself warns *"Unable to clear
 Runtime Image, subsequent launches/iterations may exhibit faster startup than production due to
@@ -81,9 +82,9 @@ The run that first completed all four tests (MIUI's install dialog answered by
 
 | benchmark | result | what it is |
 |---|---|---|
-| `coldStart` | **574.17 ms** P50 (min 553.95, max 730.19) | time to initial display — the fourth measurement of the same thing, alongside 615 / 532 / 560 ms |
+| `coldStart` | **519.5 ms** median (min 503.6, max 544.7) | time to initial display — the fifth measurement of the same thing, alongside 615 / 532 / 560 / 574 ms |
-| `openAFolderOfAThousandFiles` | **14 frames** P50 (min 13, max 17) | frames drawn from the folder tap until `file-0.txt` is on screen |
+| `openAFolderOfAThousandFiles` | **14 frames** median (min 11, max 17) | frames drawn from the folder tap until `file-0.txt` is on screen |
-| `scrollAFolderOfAThousandFiles` | **98 frames** P50 (min 93, max 100) | frames drawn across four swipes through the folder |
+| `scrollAFolderOfAThousandFiles` | **94 frames** median (min 48, max 96) | frames drawn across four swipes through the folder |
 | `searchInsideAFolderOfAThousandFiles` | **failed** | the app bar's search action is not in the accessibility tree within 20 s of the folder opening, although the same taps by hand find it immediately |
 
 **Those frame counts are not a jank metric, and the difference matters.** `FrameTimingMetric` normally
@@ -99,33 +100,48 @@ could be copied a second time, so the figures above come from parsing it once, i
 re-running on a device without an install gate is the way to confirm them. And the run still ends with a
 failing `search` test, so its number is missing rather than zero.
 
-### Why the reinstall cannot be answered by the app
+### The install moved out of the test — that is the fix
 
-macrobenchmark installs its target before measuring it, and on this phone MIUI puts that install behind
+The problem was never the dialog; it was *who* installs. `CompilationMode.None()` reports
-`com.miui.securitycenter/.permcenter.install.AdbInstallActivity`. The test cannot answer it itself:
+`shouldReset = true`, and that is what makes `resetAndCompile` uninstall the app and reinstall it with
-`CompilationMode.installPackageFromPaths` builds `pm install -t …` and runs it through
+`pm install -t --skip-verification` before every single test. On this phone that install is intercepted by
-`Shell.executeScriptCaptureStdout`, so the call blocks until the child exits and no test code is running
+`com.miui.securitycenter/.permcenter.install.AdbInstallActivity`, which the test cannot answer: the install
-while the dialog is up.
+call blocks until `pm` exits, so no test code is running while the dialog is up.
 
-`tools/benchmark-watch-install.sh` answers it from the host instead — but it is a workaround with a cost,
+`CompilationMode.Ignore()` reports `shouldReset = **false**` — the library installs nothing and measures
-and it is not part of the benchmark workflow for that reason: MIUI keeps `mCurrentFocus` pointing at the
+what is there. So the APK is installed once, from the host, and the run contains no install at all:
-installer for a moment after the window has been removed, so a tap sent on that signal can land on the
-home screen underneath (observed: it opened Telegram). The script now reads the button off the screen
-before every tap and refuses to tap when the installer is not focused, which narrows the race to the
-handful of milliseconds between that check and the tap — it does not close it. MIUI's own **Install via
-USB** setting is what removes the dialog; with it on, installs need no tap at all.
 
-Three further measurements about that dialog, kept because they explain the workaround's shape:
+```bash
+./tools/benchmark-fixture.sh                                    # /sdcard/AABenchmark, 1000 empty files
+adb install -r app/build/outputs/apk/benchmark/app-benchmark.apk
+adb install -r benchmark/build/outputs/apk/benchmark/benchmark-benchmark.apk
+adb shell am instrument -w -e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark \
+    de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner
+```
 
-- It is `FLAG_SECURE`: a `screencap` taken while `mCurrentFocus` was the installer shows the home screen
+Measured: with that command the four tests run with **zero installs** — `tools/benchmark-watch-install.sh`
-  behind it, which is why screenshots of "the dialog" show nothing.
+logs heartbeats and nothing else — and three of them pass. `Ignore()` applies no compilation profile either,
+so the numbers still describe code that has never run; a freshly installed APK has no profile to begin with.
+Running through Gradle still works, but Gradle's own install steps happen before the tests and are gated
+too, which is why `am instrument` is the documented path.
+
+`tools/benchmark-watch-install.sh` is kept for the one thing it is good at: the two host-side `adb install`
+commands above. It reads the Install button off the screen and taps only while the installer is focused. It
+is *not* a solution for a run — MIUI keeps `mCurrentFocus` on the installer for a moment after the window is
+gone, so a tap sent on that signal can land on the home screen behind it (observed: it opened Telegram).
+
+Why the dialog could not simply be answered from the test, kept because it explains the shape of all this:
+
+- The dialog is `FLAG_SECURE`: a `screencap` taken while `mCurrentFocus` was the installer shows the home
+  screen behind it, which is why screenshots of "the dialog" show nothing.
 - Its lifetime is not stable. One 0.1 s focus trace caught it as
-  `13:51:27.70 AdbInstallActivity` → `13:51:28.38 null` — too short to read, let alone tap — while a later
+  `13:51:27.70 AdbInstallActivity` → `13:51:28.38 null` — too short to read, let alone tap — while another
-  trace showed the same window focused for 8.5 s, long enough for a dump and a tap. Presumably the
+  showed the same window focused for 8.5 s.
-  difference is whether MIUI is waiting for an answer or merely announcing the install.
 - `pm install` started from an *app* uid does not even reach the gate: MIUI's `XSpaceManagerServiceImpl`
   throws `SecurityException: You either need MANAGE_USERS or CREATE_USERS permission to: query users`.
-  The rule's install therefore runs through the shell, which is the path MIUI intercepts.
+  The rule's install therefore runs through the shell, the path MIUI intercepts.
+- `MIUI`'s SecurityCenter cannot be disabled to get out of the way: `pm disable-user` answers
+  `SecurityException: Cannot disable miui core packages`.
 
 ### The four switches that make an unattended run possible at all
 
@@ -140,13 +156,13 @@ Three further measurements about that dialog, kept because they explain the work
    `app-debug.apk`, so the numbers would have been debug numbers. The type also has to
    `initWith(getByName("debug"))`: a declared type starts unsigned, and an unsigned test APK fails with
    `INSTALL_PARSE_FAILED_NO_CERTIFICATES`.
-3. **The storage grant is walked through the UI, not set with `appops`.** The rule uninstalls and
+3. **The storage grant survives because nothing uninstalls any more.** An uninstall clears app-ops, so
-   reinstalls the app, and an uninstall clears app-ops, so a grant applied by the fixture script is gone
+   while the rule was reinstalling the app, a grant applied by the fixture script was gone by the time the
-   by the time the app starts. `appops set` from inside the test does not work either — even under
+   app started — and `appops set` from inside the test does not work either (even under
-   `adoptShellPermissionIdentity`, the command fails to stderr (which `executeShellCommand` does not
+   `adoptShellPermissionIdentity` it fails to stderr, which `executeShellCommand` does not return, and
-   return) and `appops get` still reads `No operations`. What does work is tapping what a user taps:
+   `appops get` still reads `No operations`). With `Ignore()` the grant lands once and stays.
-   the app's "Grant full access" button, then the platform's consent, then leaving the all-files page.
+   `grantStorageAccess` in `BrowserBenchmark` still walks the app's own "Grant full access" flow when the
-   `grantStorageAccess` in `BrowserBenchmark` does exactly that, in `setupBlock`, unmeasured.
+   app comes up without access, which costs nothing when it is not needed.
 4. **The fixture is created through adb**, because `UiAutomation.executeShellCommand` runs with the *test
    app's* uid and cannot write `/sdcard`.
 
@@ -154,24 +170,24 @@ Three further measurements about that dialog, kept because they explain the work
 
 ```bash
 ./tools/benchmark-fixture.sh                                     # /sdcard/AABenchmark, 1000 empty files
-./tools/benchmark-watch-install.sh --seconds 900 & # answers MIUI's install dialog if it stays up
+adb install -r app/build/outputs/apk/benchmark/app-benchmark.apk  # the build under measure
-./gradlew :benchmark:connectedBenchmarkAndroidTest
+adb install -r benchmark/build/outputs/apk/benchmark/benchmark-benchmark.apk
+adb shell am instrument -w -e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark \
+    de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner
 ```
 
-`coldStart` alone can be run with `-Pandroid.testInstrumentationRunnerArguments.class=…BrowserBenchmark#coldStart`.
+One test can be run by naming it: `-e class …BrowserBenchmark#coldStart`. The numbers are printed in the
-The results land in `benchmark/build/outputs/androidTest-results/connected/benchmark/`, and — worth
+instrumentation output itself, and the per-test JSON and Perfetto traces are left in
-knowing, because it cost an hour — the per-test JSON that holds the numbers is written to
+`/sdcard/Android/media/de.softwareapp_hb.filemanager.benchmark/` — they used to be deleted with the test app
-`/sdcard/Android/media/<test-pkg>/additional_test_output/` and **deleted together with the test app when
+when a run ended, which is why the earlier numbers had to be copied mid-run.
-the run ends**, often before Gradle has copied it. Copying it in a loop while the run is in progress is
-what recovered the numbers above.
 
 ## Not measured yet, and why
 
 | number | status |
 |---|---|
-| Folder open with 1 000 files | **14 frames** P50 — measured, but as a frame *count*, not a jank figure |
+| Folder open with 1 000 files | **14 frames** median — measured, but as a frame *count*, not a jank figure |
-| Folder open with 5 000 / 20 000 files | not measured: the fixture script counts to 1 000, and every larger run needs another install cycle |
+| Folder open with 5 000 / 20 000 files | not measured: the fixture script counts to 1 000 |
-| Scrolling that folder | **98 frames** P50 over four swipes — again a frame count |
+| Scrolling that folder | **94 frames** median over four swipes — again a frame count |
 | Typing into search | **failed** in the automated run: the app bar's search action is not in the accessibility tree within 20 s of the folder opening, although the same taps by hand find it immediately |
 | Thumbnail paint while scrolling a photo grid | not measured: no test for it yet, and no frame durations on this device |