coldStart measures 532 ms median (min 499, max 805) on the benchmark build - R8, not debuggable, profileable, CompilationMode.None - against 1814 ms on the debug build, which is the honest illustration of why a debug number must never be published as the app's performance. The tool also warns that it cannot clear the runtime image on this Android version, so the figure is optimistic by an unmeasured amount; that is in PERFORMANCE.md next to the number rather than in a footnote.
The three interaction benchmarks are written and compile but have not produced a number, and getting that far took four fixes worth naming. The app under test cannot keep the app's package id (signature) and cannot use .benchmark (the test APK owns that id, INSTALL_FAILED_VERSION_DOWNGRADE); macrobenchmark needs package visibility for it or it reports NameNotFoundException for an app that pm list packages shows installed; UiAutomation.executeShellCommand runs as the test app's uid, so the fixture has to be made through adb instead - that one cost the first attempt a hang; and testBuildType has to be behind a flag or :app:testDebugUnitTest stops existing.
The workflow that works on a device with an install gate is now documented: install the two APKs by hand once, run tools/benchmark-fixture.sh, then am instrument. The Gradle task is documented as the alternative and as the reason the app under test disappeared once - it uninstalls around the run.
Gate green, map current, 386 tests unchanged.
SoftwareApp
94f85bd287