SoftwareApp
179bc2cc33
The script could report "scan started" on runs where the app was being killed, which sent me
chasing a "mid-scan death" that never happened. The mistake was the signal: dumpsys activity
services lists a service *record* from the moment startForegroundService() is called, so it says
nothing about whether startForeground() was reached - and that gap is exactly what the platform's
deadline measures. logcat's events buffer shows what really happened on every attempt:
am_crash: [..., android.app.ForegroundServiceDidNotStartInTimeException,
Context.startForegroundService() did not then call
Service.startForeground(): .../ScanService]
am_proc_died: [...]
No am_kill, no low-memory record: not memory pressure, and not a crash in the app's own code. The
script now waits for the app's notification, which only exists once startForeground() has been
called, and prints that am_crash line when the service never starts, so a failure is never
mistaken for a success again.
Also in the script:
- let the emulator settle before the run (first-boot work competes for the same CPU) and give the
process time to load its classes before asking it to start a service inside the budget, both
overridable as SETTLE_SECONDS and WARM_SECONDS,
- give up with a non-zero exit and an explicit "record this on a real device" message rather than
carrying on to record a video whose scan never started,
- read the events buffer rather than the main one when reporting the reason.
Verdict for the docs, corrected: this capture cannot be done on a KVM-less emulator, and the
previous "second failure mode" was this one misread. The last crash landed on a process that had
been alive and idle for over a minute, so no amount of warming fixes it - while on the phone the
same build scans 5,363 photos in 20 s with the notification up the whole time. The take belongs on
the phone; the script is the checklist for what it has to show.
12 KiB
12 KiB
Roadmap
Foto Dup Finder is built against a fixed milestone plan (M0–M10). This file is the
single source of truth for what is done and what remains. Before this file existed the
plan lived only in developer notes, and the CHANGELOG refers to
milestone labels (M7, M8).
Legend: ✅ done · 🚧 in progress · ⏳ not started · 💡 candidate (proposed, not scheduled)
Milestones
| # | Milestone | Status | Evidence / notes |
|---|---|---|---|
| M0 | Toolchain & scaffold | ✅ | User-local SDK, Gradle 9.3.1 + AGP 9.1.0, wrapper and version catalog, empty debug APK builds |
| M1 | Pure-Kotlin duplicate engine + JVM tests | ✅ | SHA-256 quick/full hashing, pHash + RGB dHash, union-find grouping, Exact/Balanced/Aggressive policies; tests under app/src/test |
| M2 | Data layer | ✅ | MediaStore query, coroutine-parallel hashing, incremental SQLite hash index, permission + partial-access handling |
| M3 | Compose UI | ✅ | Scan progress, grouped results with thumbnails, two-step staging, selection helpers |
| M4 | Safe deletion | ✅ | MediaStore.createDeleteRequest / createTrashRequest; legacy direct delete on Android 10 |
| M5 | Performance pass | ✅ | Size short-circuit, provider-thumbnail hashing, batched hashing (folded into M2/M3) |
| M6 | Release config & Play readiness | ✅ | R8/minify, optional signing, .aab, PLAY_RELEASE_CHECKLIST, PRIVACY.md, README |
| M7 | On-device validation & hardening | ✅ | Foreground dataSync service, one failure boundary per file, core/DecodeMath, hostile fixtures (EDGE_CASE_TESTING), versionName → 1.0.0 |
| M8 | Localisation groundwork | ✅ | Every user-visible string in res/values/strings.xml with real <plurals>, shared text/TextRes.kt; English is the only locale |
| M9 | Monetisation | ✅ | Decided: a one-time paid app — no free tier, no trial, no subscription and no in-app purchase, so there is no Billing code and no new permission. Checked before deciding: com.android.billingclient:billing:9.1.0's manifest declares only com.android.vending.BILLING, so IAP would not have added INTERNET either — but a price on the listing is the one model that changes nothing at all about the "no internet permission" claim. Base price €2.99 one-time with a 50% launch sale at €1.49, both set in the Console (Products > App pricing). €1 as the base price was considered and rejected: a sale can always be added, but a low anchor cannot be undone, and a paid app's first transaction is its only transaction. Free → paid becomes impossible after the first upload, so this had to be settled before release. Rationale, copy changes and Console steps: STORE_LISTING › App pricing, PLAY_RELEASE_CHECKLIST step 4 |
| M10 | Store assets | ✅ | tools/make-store-assets.sh renders the 512×512 icon and the 1024×500 feature graphic from the app's own vectors; tools/make-store-screenshots.sh normalises device captures to Play's 2:1 limit, one directory per form factor (phone → 1080×2160, tablet-7in → 1200×1920, tablet-10in → 1600×2560); tools/make-demo-photos.py generates the photo library the results captures are taken over, so no real photos are published; all listing copy is drafted and length-checked in STORE_LISTING. The phone set was re-shot on a real device after a real permission grant, and the two tablet sets were captured on AOSP AVDs (see the blocker below); screenshots are deliberately not committed |
Release blockers (work that is not a numbered milestone)
| Item | Detail |
|---|---|
The 5,363-photo library used to freeze at “74 / 100 · Comparing file signatures…” (a blocked syscall: zero CPU, workers sleeping). BoundedCall now abandons a stuck provider call and trips a stall breaker after three consecutive timeouts. Verified on the real test device (Android 12 / MIUI): the scan completes — 20.2 s cold, 11.0 s warm, 15 groups, 6 unreadable — where it used to hang. Covered by BoundedCallTest and ScanEngineEndToEndTest |
|
BoundedCall fix |
The provider-call deadline shipped in 1.0.1 (versionCode 2) |
| Play Console: foreground-service declaration | Declaring the dataSync type (App content › Foreground service permissions) is release-blocking. [code] the manifest already declares it, and PLAY_RELEASE_CHECKLIST step 5 now carries all four things the form asks for: the description, the user impact if the scan is deferred or interrupted, the use case (Local processing: Other — the only dataSync option that fits a local, user-initiated scan) and the video link Play requires. tools/record-fgs-demo.sh records that video — empty state → Start scan → Home → the notification in the shade while the scan continues → results — into the git-ignored store-assets/fgs-demo/; hosting it and pasting the link is still a manual step. The take itself belongs on a real device: on this machine's KVM-less AVD every scan start ended in am_crash … ForegroundServiceDidNotStartInTimeException → Force finishing activity, with no am_kill or low-memory record, i.e. software emulation is too slow to reach startForeground() inside Android's budget — and that, not a mysterious "mid-scan death", is what an earlier note here was misreading (the script's old check matched the service record, which exists before startForeground() is called, so it reported success on runs that were already dying). The evidence that this is the emulator and not the app is the phone's own history: 5,363 photos scanned in 20 s with the notification up throughout |
| Remaining Play paperwork | Console actions only, no code: the first upload + Play App Signing enrolment, the hosted privacy policy URL (the working copy is now published to Gitea as fishfacegit/foto-duplicate-finder, so the policy renders at a public, login-free URL and only the Console paste is left), Data safety, the Photo/video permissions declaration, the content rating / audience / app access / ads / export questions, and the listing copy + the three screenshot sets (phone, 7-inch and 10-inch, all captured), plus the paid-app setup: a payments profile with tax rates, the €2.99 price and the 14-day €1.49 launch sale on Products > App pricing — the price must be set before the first upload, because free → paid is not possible afterwards (M9). Keystore, versionCode, listing copy, graphics, permission audit and the signed .aab (sha256 f8e6a29a…) are all in place; rollout goes internal → closed → production. Step-by-step: PLAY_RELEASE_CHECKLIST |
The old captures showed the Photo access needed banner (an app-op is not a grant) and 03-results.png showed the developer's own library. After a real grant the set was re-shot over the generated demo library: 01-empty (no tap needed — results and selection live in ScannerViewModel, so force-stop + relaunch returns to the empty state with the grant still in place), 02-scanning (5353 / 5385), 03-results (17 groups, 54 of 5385 photos, 83.5 MB reclaimable; the headline group's 43.1 MB out-ranks the device's own 39.8 MB, so no real photo is in frame) and 04-options. Captures are taken by watching for UI text changes, because MIUI refuses synthetic taps (INJECT_EVENTS). The demo library is deleted from the test device afterwards — MediaStore 5,386 → 5,364 rows, and the 22 stale rows it leaves in the hash index are pruned by PhotoIndexStore.pruneTo on the next scan. Deliberately not captured: the staged state and the system confirmation dialog, because staging clears the demo group and would put real photos back in frame. Details: STORE_LISTING |
|
Play has a 7-inch tablet and a 10-inch tablet screenshot slot of its own (asking for ≥ 4 screenshots on large screens). Both are now captured on AOSP Android 12 AVDs — Nexus 7 2013 (1200×1920, sw600dp) and medium_tablet (1600×2560, sw800dp) — because this machine exposes no /dev/kvm: they ran in pure software emulation (-accel off -no-window -gpu swiftshader_indirect), ≈3 min to boot, with hide_error_dialogs on so SystemUI's ANR dialog could not sit in frame. Four screens per tablet over the same generated demo library: empty, scanning (20 / 22), results (2 groups · 16 of 22 photos · 43.7 MB reclaimable, 39.7 s / 53.1 s) and Options. tools/make-store-screenshots.sh is form-factor aware (FORM_FACTOR=phone|tablet-7in|tablet-10in) and the tablet form factors crop 0 px of chrome, because a tablet frame is already inside Play's 2:1 cap. Recipe: STORE_LISTING |
|
| Device-only verifications | RAW/DNG files, a real iPhone HEIC with a non-upright EXIF orientation, permission revoked mid-scan, and the Android 14 “Selected photos” flow. ✅ The index-reuse check is done: after a scan the index holds 5,363 rows, 5,357 with a perceptual hash, and a second scan runs in 11.0 s vs 20.2 s cold |
Candidate milestones (proposed, not scheduled)
| Candidate | Rationale |
|---|---|
| Colour signature | Flat-colour images (blank scans, dark UI screenshots) hash identically regardless of colour, so they are reported as mutual duplicates — see the “Known gap” in EDGE_CASE_TESTING. A small colour histogram is orthogonal to the grayscale structure hash, and the benchmark harness already has an implementation to measure against |
| Video support | Only images are scanned today (README “Known limitations”) |
| “Similar shots” section | Bursts / same-subject shots as a separate, never-pre-selected list. Needs a different algorithm and must not feed the deletion flow |
| Warm-scan speed | A second scan is faster (11.0 s) but its “similar” stage still costs ~10 s even though all 5,357 perceptual hashes are cached — the per-photo withTimeoutOrNull wrapper still runs for all 5,363 photos. Profiling this could make an unchanged library re-scan in well under a second |
Suggested order
- ✅ Done — the scan-stall hardening and the
BoundedCallfix shipped as 1.0.1 (versionCode2). Only the on-device re-check of the 5,363-photo scan remains. - Finish the Play Console paperwork and roll out through internal → closed → production.
- ✅ Done — M10 store assets: the icon and feature graphic are generated from the app's own vectors, the screenshot normaliser enforces Play's 2:1 limit, the listing copy is drafted and length-checked, and the demo library that keeps real photos out of the results capture is generated and validated. The re-shoot after a real permission grant is done as well (see blockers): four captures, normalised to 1080×2160. The normaliser is form-factor aware too, and the 7-inch and 10-inch tablet sets were captured on AOSP AVDs in software emulation (1200×1920 and 1600×2560), so all three screenshot sets now exist.
- Colour signature — the flat-colour false-positive fix.
- ✅ Done — M9 monetisation: decided as a one-time paid app (€2.99, launched with a 14-day 50% sale at €1.49), which needs no code at all. The remaining work is Console-only — a payments profile, the price and the launch sale on Products > App pricing — and the price must happen before the first upload, because free → paid stops being possible once the app has been offered for free.
Deliberately deferred: video support and the “similar shots” section.