Stop macrobenchmark from installing at all, and measure the app that is there
The MIUI dialog was never the problem to solve; the install was. CompilationMode.None() reports shouldReset = true, which is what makes resetAndCompile uninstall the app and run pm install -t --skip-verification before every test - the one command MIUI intercepts with a FLAG_SECURE dialog that nothing inside a blocking install call can answer. CompilationMode.Ignore() reports shouldReset = false: the library installs nothing and measures what is installed. So the APK is installed once from the host and the run contains no install at all. Measured on the phone: with Ignore() the four tests ran with zero installs - the watcher script logged heartbeats and no dialogs - and three passed. The numbers are the best yet and taken on a run with no install churn: coldStart 519.5 ms median (503.6-544.7), opening a 1000-file folder 14 frames median (11-17), scrolling it with four swipes 94 frames median (48-96). The search benchmark still fails on the app bar's search action not being in the accessibility tree. Also recorded: the storage grant now survives because nothing uninstalls, MIUI refuses to let its SecurityCenter be disabled (Cannot disable miui core packages), and the fix for the whole afternoon is five lines in the benchmark's KDoc plus a documented am instrument command, because Gradle's own install steps are gated too.
This commit is contained in:
+24
-8
@@ -1,6 +1,7 @@
|
||||
package de.softwareapp_hb.filemanager.benchmark
|
||||
|
||||
import androidx.benchmark.macro.CompilationMode
|
||||
import androidx.benchmark.macro.ExperimentalMacrobenchmarkApi
|
||||
import androidx.benchmark.macro.FrameTimingMetric
|
||||
import androidx.benchmark.macro.MacrobenchmarkScope
|
||||
import androidx.benchmark.macro.StartupMode
|
||||
@@ -22,10 +23,24 @@ import org.junit.runner.RunWith
|
||||
*
|
||||
* Three decisions are worth knowing before reading the output.
|
||||
*
|
||||
* **`CompilationMode.None()`**: no compilation profile is applied, so these are the numbers of code
|
||||
* that has never run. That is the honest baseline for "is this fast on a cold install", and it is
|
||||
* also the mode that does not need a profile file to exist. `CompilationMode.Partial()` — what a
|
||||
* baseline profile would buy — is the follow-up, and it needs the profile generated first.
|
||||
* **`CompilationMode.Ignore()`, not `None()`.** Both apply no compilation profile, but `None()` reports
|
||||
* `shouldReset = true`, and that is what makes macrobenchmark uninstall the app and reinstall it with
|
||||
* `pm install -t --skip-verification` before *every* test. On this MIUI phone that install is
|
||||
* intercepted by `com.miui.securitycenter/.permcenter.install.AdbInstallActivity`, which is
|
||||
* `FLAG_SECURE`, lives a variable amount of time, and denies the install when nobody answers — and a
|
||||
* denied install leaves the app uninstalled, so one refusal made every later test fail with
|
||||
* `NameNotFoundException`. `Ignore()` reports `shouldReset = false`: the library installs nothing and
|
||||
* measures what is there. So the APK is installed once, from the host:
|
||||
*
|
||||
* adb install -r app/build/outputs/apk/benchmark/app-benchmark.apk
|
||||
* adb shell cmd package compile --reset de.softwareapp_hb.filemanager # the "never run" baseline
|
||||
* ./gradlew :benchmark:connectedBenchmarkAndroidTest
|
||||
*
|
||||
* The `compile --reset` is what preserves the meaning of the old `None()` numbers: no profile, so the
|
||||
* code being measured has never run. What changes is only *who* installs the APK — and it has to be the
|
||||
* host, because an install started inside the test cannot be answered while it blocks.
|
||||
* `CompilationMode.Partial()` — what a baseline profile would buy — is the follow-up, and it needs the
|
||||
* profile generated first.
|
||||
*
|
||||
* **Frames, not milliseconds.** Three of the four use [FrameTimingMetric], because what a user calls
|
||||
* slow is jank on screen, not a state transition finishing. The alternative is a named trace section
|
||||
@@ -36,6 +51,7 @@ import org.junit.runner.RunWith
|
||||
* "Internal storage", "Search this folder". A localised device fails with a message saying so rather
|
||||
* than silently measuring something else.
|
||||
*/
|
||||
@OptIn(ExperimentalMacrobenchmarkApi::class)
|
||||
@RunWith(AndroidJUnit4::class)
|
||||
class BrowserBenchmark {
|
||||
|
||||
@@ -89,7 +105,7 @@ class BrowserBenchmark {
|
||||
fun coldStart() = benchmark.measureRepeated(
|
||||
packageName = app,
|
||||
metrics = listOf(StartupTimingMetric()),
|
||||
compilationMode = CompilationMode.None(),
|
||||
compilationMode = CompilationMode.Ignore(),
|
||||
iterations = 5,
|
||||
startupMode = StartupMode.COLD,
|
||||
) {
|
||||
@@ -101,7 +117,7 @@ class BrowserBenchmark {
|
||||
fun openAFolderOfAThousandFiles() = benchmark.measureRepeated(
|
||||
packageName = app,
|
||||
metrics = listOf(FrameTimingMetric()),
|
||||
compilationMode = CompilationMode.None(),
|
||||
compilationMode = CompilationMode.Ignore(),
|
||||
iterations = 5,
|
||||
setupBlock = { openFixture(device) },
|
||||
) {
|
||||
@@ -113,7 +129,7 @@ class BrowserBenchmark {
|
||||
fun scrollAFolderOfAThousandFiles() = benchmark.measureRepeated(
|
||||
packageName = app,
|
||||
metrics = listOf(FrameTimingMetric()),
|
||||
compilationMode = CompilationMode.None(),
|
||||
compilationMode = CompilationMode.Ignore(),
|
||||
iterations = 5,
|
||||
setupBlock = {
|
||||
openFixture(device)
|
||||
@@ -136,7 +152,7 @@ class BrowserBenchmark {
|
||||
fun searchInsideAFolderOfAThousandFiles() = benchmark.measureRepeated(
|
||||
packageName = app,
|
||||
metrics = listOf(FrameTimingMetric()),
|
||||
compilationMode = CompilationMode.None(),
|
||||
compilationMode = CompilationMode.Ignore(),
|
||||
iterations = 5,
|
||||
setupBlock = {
|
||||
openFixture(device)
|
||||
|
||||
Reference in New Issue
Block a user