Record what actually blocks the interaction benchmarks: the reinstall clears the grant
The diagnosis is now precise and it is not the installer dialog. Macrobenchmark's rule uninstalls the app and reinstalls it before measuring; an uninstall clears app-ops, so the MANAGE_EXTERNAL_STORAGE grant that tools/benchmark-fixture.sh applies is gone by the time the test navigates. appops get reads No operations after a run, the app correctly shows its choose-a-folder fallback, and the volume row the test looks for is not on screen - which is exactly the assertion that fired. No test can work around it: granting an app-op needs the shell uid, and UiAutomation.executeShellCommand runs as the test app. So the three interaction benchmarks cannot run as macrobenchmark tests on any device where this app needs the grant. PERFORMANCE.md now says that, names the two paths that would give the numbers (a plain instrumented test that avoids the rule, or a benchmark flow that needs no grant), and says what each costs. The cold-start number stands.
This commit is contained in:
+21
-11
@@ -90,18 +90,28 @@ The workflow:
|
||||
```
|
||||
|
||||
**The Gradle runner is required, not preferred.** Macrobenchmark's rule uninstalls the app at the start
|
||||
of a run and reinstalls it before measuring. On this device *that reinstall* is what MIUI refuses, so the
|
||||
run leaves the app uninstalled and then fails with `NameNotFoundException` for an app that was there a
|
||||
second earlier — seen twice through `am instrument` and once through `connectedDebugAndroidTest`, with
|
||||
`Benchmark: Uninstalling de.softwareapp_hb.filemanager` in logcat and the package gone from
|
||||
`pm list packages` afterwards.
|
||||
of a run and reinstalls it before measuring. That alone is survivable; what is not is the side effect:
|
||||
**an uninstall clears the app's app-ops**, so the `MANAGE_EXTERNAL_STORAGE` grant from
|
||||
`tools/benchmark-fixture.sh` is gone by the time the test navigates. The app then correctly shows its
|
||||
"choose a folder" fallback, the volume row is not on the screen, and the assertion says so. Checked
|
||||
directly — `appops get … MANAGE_EXTERNAL_STORAGE` reads `No operations` after a run that passed through
|
||||
the reinstall.
|
||||
|
||||
The fix is a device setting rather than code: *Developer options → Install via USB* — and on MIUI usually
|
||||
*MIUI optimization off* as well — so that an install raised by the benchmark is allowed without a prompt.
|
||||
Until that is on, **the interaction benchmarks cannot run on this phone**, and the cold-start number above
|
||||
stands alone. `tools/benchmark-fixture.sh` is still needed either way: the benchmark build has no storage
|
||||
grant of its own, and `UiAutomation.executeShellCommand` (the only shell a test has) runs with the test
|
||||
app's uid and cannot write `/sdcard`.
|
||||
There is no way around that from inside the test: granting an app-op needs the shell's uid, and
|
||||
`UiAutomation.executeShellCommand` runs with the *test app's* uid. So on a device where this app needs
|
||||
that grant — which is every device, for the browser screen — **the three interaction benchmarks cannot run
|
||||
as macrobenchmark tests here**, and the cold-start number above stands alone.
|
||||
|
||||
Two paths would give the numbers, and neither is a small change:
|
||||
|
||||
- **A plain instrumented test instead of a macrobenchmark one** (no rule, so no reinstall): install the
|
||||
benchmark build once, grant, then navigate with UiAutomator and read frame durations from
|
||||
`FrameMetrics`. It measures the release-like build, but it is our own measurement code rather than a
|
||||
maintained library, and `FrameTimingMetric` exists precisely so nobody has to write that.
|
||||
- **An app that does not need the grant for the flow being measured** — for instance a benchmark
|
||||
build that browses its own external files directory, which needs no permission. That measures a
|
||||
different screen than the one users open, so it answers a narrower question than "how fast does
|
||||
opening a 1 000-file folder feel".
|
||||
|
||||
## Not measured yet, and why
|
||||
|
||||
|
||||
Reference in New Issue
Block a user