SoftwareApp
179bc2cc33
The script could report "scan started" on runs where the app was being killed, which sent me
chasing a "mid-scan death" that never happened. The mistake was the signal: dumpsys activity
services lists a service *record* from the moment startForegroundService() is called, so it says
nothing about whether startForeground() was reached - and that gap is exactly what the platform's
deadline measures. logcat's events buffer shows what really happened on every attempt:
am_crash: [..., android.app.ForegroundServiceDidNotStartInTimeException,
Context.startForegroundService() did not then call
Service.startForeground(): .../ScanService]
am_proc_died: [...]
No am_kill, no low-memory record: not memory pressure, and not a crash in the app's own code. The
script now waits for the app's notification, which only exists once startForeground() has been
called, and prints that am_crash line when the service never starts, so a failure is never
mistaken for a success again.
Also in the script:
- let the emulator settle before the run (first-boot work competes for the same CPU) and give the
process time to load its classes before asking it to start a service inside the budget, both
overridable as SETTLE_SECONDS and WARM_SECONDS,
- give up with a non-zero exit and an explicit "record this on a real device" message rather than
carrying on to record a video whose scan never started,
- read the events buffer rather than the main one when reporting the reason.
Verdict for the docs, corrected: this capture cannot be done on a KVM-less emulator, and the
previous "second failure mode" was this one misread. The last crash landed on a process that had
been alive and idle for over a minute, so no amount of warming fixes it - while on the phone the
same build scans 5,363 photos in 20 s with the notification up the whole time. The take belongs on
the phone; the script is the checklist for what it has to show.
67 lines
12 KiB
Markdown
67 lines
12 KiB
Markdown
# Roadmap
|
||
|
||
Foto Dup Finder is built against a fixed milestone plan (**M0–M10**). This file is the
|
||
single source of truth for what is done and what remains. Before this file existed the
|
||
plan lived only in developer notes, and the [CHANGELOG](../CHANGELOG.md) refers to
|
||
milestone labels (`M7`, `M8`).
|
||
|
||
Legend: ✅ done · 🚧 in progress · ⏳ not started · 💡 candidate (proposed, not scheduled)
|
||
|
||
## Milestones
|
||
|
||
| # | Milestone | Status | Evidence / notes |
|
||
|---|---|---|---|
|
||
| M0 | Toolchain & scaffold | ✅ | User-local SDK, Gradle 9.3.1 + AGP 9.1.0, wrapper and version catalog, empty debug APK builds |
|
||
| M1 | Pure-Kotlin duplicate engine + JVM tests | ✅ | SHA-256 quick/full hashing, pHash + RGB dHash, union-find grouping, Exact/Balanced/Aggressive policies; tests under `app/src/test` |
|
||
| M2 | Data layer | ✅ | MediaStore query, coroutine-parallel hashing, incremental SQLite hash index, permission + partial-access handling |
|
||
| M3 | Compose UI | ✅ | Scan progress, grouped results with thumbnails, two-step staging, selection helpers |
|
||
| M4 | Safe deletion | ✅ | `MediaStore.createDeleteRequest` / `createTrashRequest`; legacy direct delete on Android 10 |
|
||
| M5 | Performance pass | ✅ | Size short-circuit, provider-thumbnail hashing, batched hashing (folded into M2/M3) |
|
||
| M6 | Release config & Play readiness | ✅ | R8/minify, optional signing, `.aab`, [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md), [PRIVACY.md](../PRIVACY.md), README |
|
||
| M7 | On-device validation & hardening | ✅ | Foreground `dataSync` service, one failure boundary per file, `core/DecodeMath`, hostile fixtures ([EDGE_CASE_TESTING](EDGE_CASE_TESTING.md)), `versionName` → 1.0.0 |
|
||
| M8 | Localisation groundwork | ✅ | Every user-visible string in `res/values/strings.xml` with real `<plurals>`, shared `text/TextRes.kt`; **English is the only locale** |
|
||
| M9 | Monetisation | ✅ | **Decided: a one-time paid app** — no free tier, no trial, no subscription and no in-app purchase, so there is **no Billing code** and no new permission. Checked before deciding: `com.android.billingclient:billing:9.1.0`'s manifest declares only `com.android.vending.BILLING`, so IAP would *not* have added `INTERNET` either — but a price on the listing is the one model that changes nothing at all about the "no internet permission" claim. Base price **€2.99** one-time with a **50% launch sale at €1.49**, both set in the Console (*Products > App pricing*). €1 as the *base* price was considered and rejected: a sale can always be added, but a low anchor cannot be undone, and a paid app's first transaction is its only transaction. Free → paid becomes impossible after the first upload, so this had to be settled before release. Rationale, copy changes and Console steps: [STORE_LISTING](STORE_LISTING.md) › App pricing, [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) step 4 |
|
||
| M10 | Store assets | ✅ | `tools/make-store-assets.sh` renders the 512×512 icon and the 1024×500 feature graphic from the app's own vectors; `tools/make-store-screenshots.sh` normalises device captures to Play's 2:1 limit, one directory per form factor (`phone` → 1080×2160, `tablet-7in` → 1200×1920, `tablet-10in` → 1600×2560); `tools/make-demo-photos.py` generates the photo library the results captures are taken over, so no real photos are published; all listing copy is drafted and length-checked in [STORE_LISTING](STORE_LISTING.md). The phone set was re-shot on a real device after a real permission grant, and the two tablet sets were captured on AOSP AVDs (see the blocker below); screenshots are deliberately not committed |
|
||
|
||
## Release blockers (work that is not a numbered milestone)
|
||
|
||
| Item | Detail |
|
||
|---|---|
|
||
| ~~Large-library scan stall~~ ✅ **fixed — verified on device** | The 5,363-photo library used to freeze at *“74 / 100 · Comparing file signatures…”* (a **blocked syscall**: zero CPU, workers sleeping). `BoundedCall` now abandons a stuck provider call **and** trips a stall breaker after three consecutive timeouts. **Verified on the real test device (Android 12 / MIUI): the scan completes — 20.2 s cold, 11.0 s warm, 15 groups, 6 unreadable — where it used to hang.** Covered by `BoundedCallTest` and `ScanEngineEndToEndTest` |
|
||
| ~~Unreleased `BoundedCall` fix~~ ✅ | The provider-call deadline shipped in **1.0.1** (`versionCode` 2) |
|
||
| **Play Console: foreground-service declaration** | Declaring the `dataSync` type (App content › Foreground service permissions) is release-blocking. **[code]** the manifest already declares it, and [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) step 5 now carries all four things the form asks for: the description, the user impact if the scan is deferred or interrupted, the use case (**Local processing: Other** — the only `dataSync` option that fits a local, user-initiated scan) and the **video link** Play requires. `tools/record-fgs-demo.sh` records that video — empty state → *Start scan* → Home → the notification in the shade while the scan continues → results — into the git-ignored `store-assets/fgs-demo/`; hosting it and pasting the link is still a manual step. The take itself belongs on a real device: on this machine's KVM-less AVD every scan start ended in `am_crash … ForegroundServiceDidNotStartInTimeException` → *Force finishing activity*, with no `am_kill` or low-memory record, i.e. software emulation is too slow to reach `startForeground()` inside Android's budget — and that, not a mysterious "mid-scan death", is what an earlier note here was misreading (the script's old check matched the service *record*, which exists before `startForeground()` is called, so it reported success on runs that were already dying). The evidence that this is the emulator and not the app is the phone's own history: 5,363 photos scanned in 20 s with the notification up throughout |
|
||
| **Remaining Play paperwork** | Console actions only, no code: the **first upload + Play App Signing enrolment**, the **hosted privacy policy URL** (the working copy is now published to Gitea as [`fishfacegit/foto-duplicate-finder`](https://gitea.softwareapp-hb.de/fishfacegit/foto-duplicate-finder), so the policy renders at a public, login-free URL and only the Console paste is left), **Data safety**, the **Photo/video permissions** declaration, the **content rating / audience / app access / ads / export** questions, and the **listing copy + the three screenshot sets** (phone, 7-inch and 10-inch, all captured), plus the **paid-app setup**: a payments profile with tax rates, the €2.99 price and the 14-day €1.49 launch sale on *Products > App pricing* — the price must be set before the first upload, because free → paid is not possible afterwards (M9). Keystore, `versionCode`, listing copy, graphics, permission audit and the signed `.aab` (`sha256 f8e6a29a…`) are all in place; rollout goes internal → closed → production. Step-by-step: [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) |
|
||
| ~~Store screenshots need one re-shoot~~ ✅ **done — verified on device** | The old captures showed the *Photo access needed* banner (an app-op is not a grant) and `03-results.png` showed the developer's own library. After a real grant the set was re-shot over the generated demo library: **01-empty** (no tap needed — results and selection live in `ScannerViewModel`, so force-stop + relaunch returns to the empty state with the grant still in place), **02-scanning** (`5353 / 5385`), **03-results** (**17 groups, 54 of 5385 photos, 83.5 MB reclaimable**; the headline group's 43.1 MB out-ranks the device's own 39.8 MB, so no real photo is in frame) and **04-options**. Captures are taken by watching for UI **text** changes, because MIUI refuses synthetic taps (`INJECT_EVENTS`). The demo library is deleted from the test device afterwards — MediaStore **5,386 → 5,364** rows, and the 22 stale rows it leaves in the hash index are pruned by `PhotoIndexStore.pruneTo` on the next scan. Deliberately not captured: the staged state and the system confirmation dialog, because staging clears the demo group and would put real photos back in frame. Details: [STORE_LISTING](STORE_LISTING.md) |
|
||
| ~~Tablet store screenshots outstanding~~ ✅ **done — captured on AVDs** | Play has a **7-inch tablet** and a **10-inch tablet** screenshot slot of its own (asking for **≥ 4** screenshots on large screens). Both are now captured on AOSP Android 12 AVDs — `Nexus 7 2013` (1200×1920, `sw600dp`) and `medium_tablet` (1600×2560, `sw800dp`) — because this machine exposes no `/dev/kvm`: they ran in **pure software emulation** (`-accel off -no-window -gpu swiftshader_indirect`), ≈3 min to boot, with `hide_error_dialogs` on so SystemUI's ANR dialog could not sit in frame. Four screens per tablet over the same generated demo library: empty, scanning (20 / 22), results (**2 groups · 16 of 22 photos · 43.7 MB reclaimable**, 39.7 s / 53.1 s) and Options. `tools/make-store-screenshots.sh` is form-factor aware (`FORM_FACTOR=phone\|tablet-7in\|tablet-10in`) and the tablet form factors crop **0 px** of chrome, because a tablet frame is already inside Play's 2:1 cap. Recipe: [STORE_LISTING](STORE_LISTING.md) |
|
||
| **Device-only verifications** | RAW/DNG files, a *real* iPhone HEIC with a non-upright EXIF orientation, permission revoked mid-scan, and the Android 14 “Selected photos” flow. ✅ The **index-reuse** check is done: after a scan the index holds **5,363 rows, 5,357 with a perceptual hash**, and a second scan runs in **11.0 s vs 20.2 s** cold |
|
||
|
||
## Candidate milestones (proposed, not scheduled)
|
||
|
||
| Candidate | Rationale |
|
||
|---|---|
|
||
| **Colour signature** | Flat-colour images (blank scans, dark UI screenshots) hash identically regardless of colour, so they are reported as mutual duplicates — see the “Known gap” in [EDGE_CASE_TESTING](EDGE_CASE_TESTING.md). A small colour histogram is orthogonal to the grayscale structure hash, and the benchmark harness already has an implementation to measure against |
|
||
| **Video support** | Only images are scanned today (README “Known limitations”) |
|
||
| **“Similar shots” section** | Bursts / same-subject shots as a separate, never-pre-selected list. Needs a different algorithm and must not feed the deletion flow |
|
||
| **Warm-scan speed** | A second scan is faster (11.0 s) but its “similar” stage still costs ~10 s even though all 5,357 perceptual hashes are cached — the per-photo `withTimeoutOrNull` wrapper still runs for all 5,363 photos. Profiling this could make an unchanged library re-scan in well under a second |
|
||
|
||
## Suggested order
|
||
|
||
1. ✅ **Done** — the scan-stall hardening and the `BoundedCall` fix shipped as **1.0.1**
|
||
(`versionCode` 2). Only the on-device re-check of the 5,363-photo scan remains.
|
||
2. Finish the Play Console paperwork and roll out through **internal → closed → production**.
|
||
3. ✅ **Done** — **M10 store assets**: the icon and feature graphic are generated from the
|
||
app's own vectors, the screenshot normaliser enforces Play's 2:1 limit, the listing copy is
|
||
drafted and length-checked, and the demo library that keeps real photos out of the results
|
||
capture is generated and validated. The re-shoot after a real permission grant is done as well
|
||
(see blockers): four captures, normalised to 1080×2160. The normaliser is form-factor aware too,
|
||
and the 7-inch and 10-inch tablet sets were captured on AOSP AVDs in software emulation
|
||
(1200×1920 and 1600×2560), so all three screenshot sets now exist.
|
||
4. **Colour signature** — the flat-colour false-positive fix.
|
||
5. ✅ **Done** — **M9 monetisation**: decided as a **one-time paid app (€2.99, launched with a 14-day
|
||
50% sale at €1.49)**, which needs no code at all. The remaining work is Console-only — a payments
|
||
profile, the price and the launch sale on *Products > App pricing* — and the price must happen
|
||
before the first upload, because free → paid stops being possible once the app has been offered
|
||
for free.
|
||
|
||
Deliberately deferred: **video support** and the **“similar shots”** section.
|