Get a real cold-start number out of the benchmark, and record what the device demanded
coldStart measures 532 ms median (min 499, max 805) on the benchmark build - R8, not debuggable, profileable, CompilationMode.None - against 1814 ms on the debug build, which is the honest illustration of why a debug number must never be published as the app's performance. The tool also warns that it cannot clear the runtime image on this Android version, so the figure is optimistic by an unmeasured amount; that is in PERFORMANCE.md next to the number rather than in a footnote. The three interaction benchmarks are written and compile but have not produced a number, and getting that far took four fixes worth naming. The app under test cannot keep the app's package id (signature) and cannot use .benchmark (the test APK owns that id, INSTALL_FAILED_VERSION_DOWNGRADE); macrobenchmark needs package visibility for it or it reports NameNotFoundException for an app that pm list packages shows installed; UiAutomation.executeShellCommand runs as the test app's uid, so the fixture has to be made through adb instead - that one cost the first attempt a hang; and testBuildType has to be behind a flag or :app:testDebugUnitTest stops existing. The workflow that works on a device with an install gate is now documented: install the two APKs by hand once, run tools/benchmark-fixture.sh, then am instrument. The Gradle task is documented as the alternative and as the reason the app under test disappeared once - it uninstalls around the run. Gate green, map current, 386 tests unchanged.
This commit is contained in:
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Prepares the device for the macrobenchmark module, and takes it back afterwards.
|
||||
#
|
||||
# ./tools/benchmark-fixture.sh # create the folder, grant storage access
|
||||
# ./tools/benchmark-fixture.sh --remove # delete the folder again
|
||||
#
|
||||
# Why this is a script rather than setup inside the benchmark: UiAutomation's executeShellCommand
|
||||
# runs with the *test app's* uid, not the shell's, so it cannot write to /sdcard and cannot grant
|
||||
# itself an app-op. The first version of the benchmark tried exactly that and failed with a
|
||||
# RuntimeException from UiAutomationConnection, leaving no fixture and a hang. Through adb the same
|
||||
# commands run as the shell user, which has the rights this needs.
|
||||
#
|
||||
# The folder is at the root of shared storage under a name that sorts before everything else, so the
|
||||
# benchmark finds its row without scrolling.
|
||||
set -euo pipefail
|
||||
|
||||
package=de.softwareapp_hb.filemanager.perf
|
||||
directory=/sdcard/AABenchmark
|
||||
count=1000
|
||||
|
||||
case "${1:-create}" in
|
||||
--remove)
|
||||
adb shell "rm -rf $directory"
|
||||
echo "benchmark-fixture: removed $directory"
|
||||
;;
|
||||
create)
|
||||
# One shell loop with builtin redirection: no fork per file, so a thousand files take a moment
|
||||
# rather than a minute.
|
||||
adb shell "rm -rf $directory; mkdir -p $directory; \
|
||||
i=0; while [ \$i -lt $count ]; do : > $directory/file-\$i.txt; i=\$((i+1)); done"
|
||||
# The benchmark build is not the debug build, so it starts with no storage grant; granting it
|
||||
# here is what lets the run be unattended. No `--uid`: that flag takes a uid, not a package
|
||||
# name, and passing a package there grants nothing while still reporting that something was
|
||||
# set — which is exactly the kind of half-truth that costs an afternoon.
|
||||
adb shell "appops set $package MANAGE_EXTERNAL_STORAGE allow"
|
||||
echo "benchmark-fixture: $count files in $directory, storage granted to $package"
|
||||
;;
|
||||
*)
|
||||
echo "usage: $0 [--remove]" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
Reference in New Issue
Block a user