diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 32c138c..b39a98b 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -90,18 +90,28 @@ The workflow: ``` **The Gradle runner is required, not preferred.** Macrobenchmark's rule uninstalls the app at the start -of a run and reinstalls it before measuring. On this device *that reinstall* is what MIUI refuses, so the -run leaves the app uninstalled and then fails with `NameNotFoundException` for an app that was there a -second earlier — seen twice through `am instrument` and once through `connectedDebugAndroidTest`, with -`Benchmark: Uninstalling de.softwareapp_hb.filemanager` in logcat and the package gone from -`pm list packages` afterwards. +of a run and reinstalls it before measuring. That alone is survivable; what is not is the side effect: +**an uninstall clears the app's app-ops**, so the `MANAGE_EXTERNAL_STORAGE` grant from +`tools/benchmark-fixture.sh` is gone by the time the test navigates. The app then correctly shows its +"choose a folder" fallback, the volume row is not on the screen, and the assertion says so. Checked +directly — `appops get … MANAGE_EXTERNAL_STORAGE` reads `No operations` after a run that passed through +the reinstall. -The fix is a device setting rather than code: *Developer options → Install via USB* — and on MIUI usually -*MIUI optimization off* as well — so that an install raised by the benchmark is allowed without a prompt. -Until that is on, **the interaction benchmarks cannot run on this phone**, and the cold-start number above -stands alone. `tools/benchmark-fixture.sh` is still needed either way: the benchmark build has no storage -grant of its own, and `UiAutomation.executeShellCommand` (the only shell a test has) runs with the test -app's uid and cannot write `/sdcard`. +There is no way around that from inside the test: granting an app-op needs the shell's uid, and +`UiAutomation.executeShellCommand` runs with the *test app's* uid. So on a device where this app needs +that grant — which is every device, for the browser screen — **the three interaction benchmarks cannot run +as macrobenchmark tests here**, and the cold-start number above stands alone. + +Two paths would give the numbers, and neither is a small change: + +- **A plain instrumented test instead of a macrobenchmark one** (no rule, so no reinstall): install the + benchmark build once, grant, then navigate with UiAutomator and read frame durations from + `FrameMetrics`. It measures the release-like build, but it is our own measurement code rather than a + maintained library, and `FrameTimingMetric` exists precisely so nobody has to write that. +- **An app that does not need the grant for the flow being measured** — for instance a benchmark + build that browses its own external files directory, which needs no permission. That measures a + different screen than the one users open, so it answers a narrower question than "how fast does + opening a 1 000-file folder feel". ## Not measured yet, and why