Initial commit: Foto Dup Finder 1.0.1
An Android duplicate-photo finder. Pure-Kotlin core (SHA-256 + dHash hashing, exact and perceptual union-find grouping, three sensitivity presets) with MediaStore access, a persistent SQLite hash index and a tiered ScanEngine behind a Jetpack Compose UI. No INTERNET permission, no ads, no analytics, no account. Ships versionCode 2 / versionName 1.0.1, carrying the scan-stall hardening (BoundedCall breaker) and verified against a 5,363-photo device library. 80 JVM unit tests, lint clean. Release paperwork is in docs/ (Play checklist, store listing, edge-case testing), the upload-ready Play graphics and screenshots in store-assets/, and the generators for those assets in tools/.
This commit is contained in:
@@ -0,0 +1,212 @@
|
||||
# Edge-case testing
|
||||
|
||||
Fixtures are generated, not committed:
|
||||
|
||||
```bash
|
||||
# text/PNG/JPEG fixtures
|
||||
python3 tools/make-edge-case-fixtures.py
|
||||
|
||||
# ...plus HEIC (needs an encoder; ImageMagick's HEIC support is read-only)
|
||||
python3 -m venv /tmp/heicvenv && /tmp/heicvenv/bin/pip install pillow-heif
|
||||
/tmp/heicvenv/bin/python tools/make-edge-case-fixtures.py
|
||||
|
||||
# push to a phone
|
||||
adb push fixtures/foto-dup-edge /sdcard/DCIM/foto-dup-edge
|
||||
```
|
||||
|
||||
Everything lands in `fixtures/foto-dup-edge/` **inside the repo**. The directory is git-ignored
|
||||
— the fixtures are regenerated on demand, so no binary test data is committed.
|
||||
|
||||
## What each fixture is for
|
||||
|
||||
| Fixture | Expected behaviour |
|
||||
|---|---|
|
||||
| `valid.jpg`, `tiny-1px.jpg`, `grayscale.jpg`, `with-alpha.png` | read normally |
|
||||
| `zero-bytes.jpg`, `truncated.jpg`, `garbage-body.jpg` | skipped, counted in "could not be read", **no crash** |
|
||||
| `renamed-png.jpg` (PNG named `.jpg`) | depends on the MIME reported by MediaStore — must not crash |
|
||||
| `no-extension` | skipped by MediaStore; must not crash if it appears |
|
||||
| `decompression-bomb.png` (declares 60 000²) | skipped, never allocates; exercises `DecodeMath` |
|
||||
| `dup-a.jpg` / `dup-b.jpg` | **one exact-duplicate group** |
|
||||
| `solid-0..5.png` | Flat colour: every one hashes *identically*, so they all belong in **one** group |
|
||||
| `exif-orientation-1..8.jpg` | Identical **pixels** with different orientation **tags**, so they *display differently*. Only the upright one groups with the untagged PNG copies; the other seven are correctly distinct. A small group here therefore **proves EXIF normalisation is applied** — if it were ignored, all eight share identical pixels and would collapse into one group with the PNGs |
|
||||
| `heic-basic.heic` | decoded and hashed like any other photo |
|
||||
| `heic-dup.heic` | byte-identical to `heic-basic` → one exact-duplicate group |
|
||||
| `heic-large.heic` (2400×1800) | exercises the subsampled fallback decode |
|
||||
| `heic-solid-0/1.heic` | ditto the PNG solids, but they should join them: flat images of *any* colour are indistinguishable to the current descriptors |
|
||||
| `scene.jpg` + `scene.heic` (one scene, two codecs) | informative: is a codec conversion still reported? |
|
||||
|
||||
## Licence
|
||||
|
||||
All fixtures are **generated locally from synthetic images**, so they carry no third-party
|
||||
licence and can be pushed, kept or discarded freely.
|
||||
|
||||
The obvious sources for ready-made HEIC samples are **not** clean:
|
||||
|
||||
- `nokiatech/heif_conformance` — the canonical HEIC test set, but the repository has **no
|
||||
LICENSE file** and the README states no terms, so it is all-rights-reserved by default.
|
||||
- `strukturag/libheif` `examples/example.heic` — the repo is LGPL (library) / MIT (example
|
||||
apps), but the **image file itself has no stated licence**.
|
||||
- Wikimedia Commons — **hosts no HEIC files** (not an accepted upload format).
|
||||
|
||||
That is why the HEIC fixtures are encoded here with `pillow-heif` instead.
|
||||
|
||||
## Known gap this fixture set exposed
|
||||
|
||||
**Flat-colour images are indistinguishable to the current descriptors.** A uniform image has no
|
||||
structure, so its pHash is always `1` and its RGB dHash is always `0` — regardless of the colour.
|
||||
A black screenshot, a white scan and a grey solid therefore all hash *identically* and get reported
|
||||
as mutual duplicates.
|
||||
|
||||
This is a genuine false-positive risk on real libraries (dark UI screenshots, blank scans). The fix
|
||||
is to add the colour dimension the descriptors deliberately discard — a small colour histogram is
|
||||
orthogonal to a grayscale structural hash and would separate them (see the benchmark harness, which
|
||||
already has a colour-histogram implementation).
|
||||
|
||||
## Running the grouping logic on a desktop
|
||||
|
||||
The duplicate engine is pure Kotlin, so the algorithm can be exercised without a phone. Only
|
||||
MediaStore, the SQLite index and the notification need Android.
|
||||
|
||||
```bash
|
||||
# normalise EXIF orientation with Pillow, then fingerprint and group in Kotlin
|
||||
python3 tools/normalise-for-local-grouping.py fixtures/foto-dup-edge build/fotonorm
|
||||
./gradlew :app:testDebugUnitTest --tests '*LocalGroupingTest*' \
|
||||
-DfotoLocal.dir=/tmp/fotonorm --rerun-tasks
|
||||
```
|
||||
|
||||
(`--rerun-tasks` because Gradle does not always notice a changed `-D` value on the test task.)
|
||||
|
||||
Faithful: hashing, the adaptive threshold, banded candidate generation, the pHash∧RGB conjunction,
|
||||
union-find clustering and the resulting groups.
|
||||
Not faithful: MediaStore's view of what is indexed, the provider's thumbnail decode, and the
|
||||
SQLite hash cache.
|
||||
|
||||
The last two matter — see below.
|
||||
|
||||
### Why the device and a desktop run can disagree on flat images
|
||||
|
||||
Local run of the fixture set: **all eight solids in one group**, as the algorithm demands.
|
||||
On a real device the same eight came back as 3 + 2 + 2.
|
||||
|
||||
Reproduced locally by feeding the harness *thumbnail-sized JPEGs* (96×72, q75) instead of the
|
||||
lossless solids: the result becomes **3 + 2 + 2**, matching the device.
|
||||
|
||||
Cause: JPEG quantises the DC coefficient, so a one-unit colour difference between neighbouring
|
||||
solids falls below the quantisation step and they become effectively identical. On a device the
|
||||
perceptual hash is taken from the provider's **lossy thumbnail**, so on a flat image it is
|
||||
hash-of-quantisation-noise — there is no signal left to group by.
|
||||
|
||||
Two consequences:
|
||||
|
||||
1. Flat images (blank scans, dark UI screenshots) are grouped arbitrarily. **A colour signature is
|
||||
the fix** — it is the one dimension the current descriptors discard.
|
||||
2. Because the hash comes from the provider's thumbnail, it depends on the thumbnail's
|
||||
re-encoding, not purely on the file. Real near-duplicates survive this (they are close
|
||||
anyway), but the margin is thinner than a self-decoded hash would give.
|
||||
|
||||
## Running the scan pipeline on the JVM (Robolectric)
|
||||
|
||||
`ScanEngineEndToEndTest` drives the **real** pipeline — MediaStore query, head/tail signature,
|
||||
SHA-256, SQLite index and grouper — against `FakeMediaStoreProvider`, a stand-in ContentProvider
|
||||
registered under the MediaStore authority and backed by real files on disk.
|
||||
|
||||
```bash
|
||||
./gradlew :app:testDebugUnitTest --tests '*ScanEngineEndToEndTest*'
|
||||
```
|
||||
|
||||
**What it covers:** query and projection handling, the head/tail read, the exact-duplicate path over
|
||||
genuine bytes, the index across two runs, unreadable-file reporting, and that the scan **runs to
|
||||
completion instead of stalling** — the failure that froze on a device.
|
||||
|
||||
**What it cannot cover:** perceptual hashing. Robolectric's `ShadowBitmapFactory` does not decode
|
||||
real pixel data, so every fingerprint is `null` and all files are reported as unreadable. The test
|
||||
asserts that reality explicitly rather than pretending otherwise; near-duplicate behaviour has to
|
||||
come from `bench/` or from a device.
|
||||
|
||||
### JDK 17+ needs two different module flags
|
||||
|
||||
Robolectric reflects into JDK internals, and on a modern JDK that fails unless the test JVM is
|
||||
opened up. Two *distinct* flags are needed, and the second is easy to miss:
|
||||
|
||||
```
|
||||
--add-opens=java.base/java.io=ALL-UNNAMED # FileDescriptor fields
|
||||
--add-exports=java.base/jdk.internal.access=ALL-UNNAMED # SharedSecrets accessor
|
||||
```
|
||||
|
||||
The second must be an **export**, not an open: Robolectric calls
|
||||
`jdk.internal.access.SharedSecrets#getJavaIOFileDescriptorAccess` reflectively, and the failure
|
||||
otherwise surfaces as an opaque `IllegalAccessException` at `Reflection.java:400`, wrapped in
|
||||
*"Failed to interact with raw FileDescriptor internals; perhaps JRE has changed?"*.
|
||||
|
||||
## What cannot be generated locally
|
||||
|
||||
- **HEIC with an EXIF orientation tag.** `pillow-heif` accepts `exif=` on save but the
|
||||
orientation tag reads back as `1` regardless, so a generated fixture would misrepresent what
|
||||
it tests. This case needs a **real camera/iPhone HEIC** with a non-upright orientation.
|
||||
- **HEIC from a real encoder** in general — worth spot-checking one real iPhone file.
|
||||
- **Burst sequences / same-scene shots.** The one behaviour the benchmark cannot model, and the
|
||||
most likely source of false groups. Needs real photos.
|
||||
- **Permission revoked mid-scan**, and the Android 14 "Selected photos" flow: device only.
|
||||
|
||||
## Driving a scan on a device without touch input
|
||||
|
||||
The test device (Xiaomi M2101K7AG, Android 12 / MIUI, serial `2a2ae895`) denies the usual
|
||||
automation, so a scan has to be driven from the app side instead:
|
||||
|
||||
| Usual approach | Result on this device |
|
||||
|---|---|
|
||||
| `adb shell input tap/keyevent` | `SecurityException: INJECT_EVENTS` — MIUI blocks it, and the toggle that would allow it has no adb-reachable switch |
|
||||
| `adb shell pm grant <pkg> READ_EXTERNAL_STORAGE` | `SecurityException: Neither user 2000 nor current process has GRANT_RUNTIME_PERMISSIONS` |
|
||||
| `adb install -g` | same refusal |
|
||||
|
||||
Two **debug-only** hooks exist for this, under `app/src/debug/`. Both are absent from the
|
||||
release build — re-check with the audit command in `PLAY_RELEASE_CHECKLIST.md` § 5.
|
||||
|
||||
### Starting a scan
|
||||
|
||||
```bash
|
||||
adb shell am broadcast -a de.softwareapp_hb.fotodup.finder.debug.START_SCAN \
|
||||
-n de.softwareapp_hb.fotodup.finder.debug/de.softwareapp_hb.fotodup.finder.debug.DebugScanTrigger
|
||||
```
|
||||
|
||||
Add `--es policy exact|balanced|aggressive` to override the default (`balanced`). The receiver
|
||||
logs what it did and the resulting `ScanState` under the `FotoDupDebug` tag, so progress **and**
|
||||
the final timings are readable without the screen:
|
||||
|
||||
```bash
|
||||
adb logcat -s FotoDupDebug
|
||||
# poll the current state at any time:
|
||||
adb shell am broadcast -a de.softwareapp_hb.fotodup.finder.debug.REPORT_STATE \
|
||||
-n de.softwareapp_hb.fotodup.finder.debug/de.softwareapp_hb.fotodup.finder.debug.DebugScanTrigger
|
||||
```
|
||||
|
||||
### Granting photo access
|
||||
|
||||
`MANAGE_EXTERNAL_STORAGE` is the one storage permission that *can* be granted from adb, which is
|
||||
why the **debug** manifest declares it:
|
||||
|
||||
```bash
|
||||
adb shell appops set de.softwareapp_hb.fotodup.finder.debug MANAGE_EXTERNAL_STORAGE allow
|
||||
# ...run the scan, then put it back:
|
||||
adb shell appops set de.softwareapp_hb.fotodup.finder.debug MANAGE_EXTERNAL_STORAGE default
|
||||
```
|
||||
|
||||
The app's own banner still reads *“Photo access needed”*, because `checkSelfPermission` is
|
||||
unaffected by an appops grant. That is expected: it only affects the on-screen prompt, not the scan.
|
||||
|
||||
### Attaching a debugger for thread stacks
|
||||
|
||||
`jdb` attaches over JDWP to the debug build:
|
||||
|
||||
```bash
|
||||
adb forward tcp:8700 jdwp:$(adb shell pidof de.softwareapp_hb.fotodup.finder.debug | tr -d '\r')
|
||||
printf 'suspend\nwhere all\nquit\n' | jdb -attach localhost:8700
|
||||
```
|
||||
|
||||
Two things learned the hard way:
|
||||
|
||||
- `print` needs a **selected frame**. With no current frame even `java.lang.Class` reports
|
||||
"Name unknown", so expressions cannot be evaluated until a breakpoint has been reached.
|
||||
- Leaving the main thread suspended for more than a few seconds **during an activity launch**
|
||||
gets the process killed by the launch timeout. Evaluate and `cont` quickly.
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
# Google Play release checklist
|
||||
|
||||
Work top to bottom. Items marked **[code]** are already handled in this repository;
|
||||
the rest are things you must do in the Play Console / with your own accounts.
|
||||
|
||||
## 1. Identity and signing
|
||||
|
||||
- [x] Final **application id** chosen: `de.softwareapp_hb.fotodup.finder`
|
||||
(`applicationId` in `app/build.gradle.kts`). It can **never be changed after the
|
||||
first release**, so treat it as locked. `namespace` and the Kotlin package of
|
||||
the same name are aligned with it; debug builds append `.debug`.
|
||||
- [ ] Generate an **upload keystore** (keep it safe; if you lose it you cannot
|
||||
publish updates):
|
||||
```bash
|
||||
keytool -genkeypair -v -keystore upload-keystore.jks \
|
||||
-alias upload -keyalg RSA -keysize 4096 -validity 10000
|
||||
```
|
||||
- [ ] Copy `keystore.properties.example` to `keystore.properties` and fill it in.
|
||||
**[code]** The Gradle build picks it up automatically and signs release
|
||||
builds; without it the release APK stays unsigned.
|
||||
- [ ] Enable **Play App Signing** in the Play Console and upload the certificate of
|
||||
your *upload* key.
|
||||
- [ ] Store the keystore + passwords in a password manager / secure vault. Back it up.
|
||||
|
||||
## 2. Versioning
|
||||
|
||||
- [ ] Set `versionName` (user-visible, e.g. `1.0.0`) **[code]** in
|
||||
`app/build.gradle.kts`.
|
||||
- [ ] Increment `versionCode` for **every** upload.
|
||||
|
||||
## 3. Build the release artefacts
|
||||
|
||||
```bash
|
||||
./gradlew :app:bundleRelease # -> app/build/outputs/bundle/release/app-release.aab
|
||||
./gradlew :app:assembleRelease # -> APK for local testing
|
||||
```
|
||||
|
||||
- [ ] Upload the **`.aab`** (not the APK) to the Play Console.
|
||||
- [ ] Keep `app/build/outputs/mapping/release/mapping.txt` for every release so you
|
||||
can de-obfuscate crash reports (upload it under *Release > App bundle explorer*).
|
||||
|
||||
## 4. Store listing
|
||||
|
||||
Ready-to-paste copy and the generated graphics live in
|
||||
[docs/STORE_LISTING.md](STORE_LISTING.md) (milestone M10). Everything below is a Console
|
||||
action except the items marked **[code]**.
|
||||
|
||||
- [ ] App name (≤ 30 chars): "Foto Dup Finder".
|
||||
- [ ] Short description (≤ 80 chars) — two options are drafted and length-checked in
|
||||
STORE_LISTING.md.
|
||||
- [ ] Full description (≤ 4000 chars) — drafted and length-checked in STORE_LISTING.md.
|
||||
- [x] **App icon** 512×512 PNG **[code]** — `tools/make-store-assets.sh` renders it from the
|
||||
same vectors as the adaptive icon in `res/drawable/ic_launcher_*`.
|
||||
- [x] **Feature graphic** 1024×500 PNG **[code]** — same script.
|
||||
- [ ] At least 2 phone screenshots (up to 8). Three are captured (empty state, scanning,
|
||||
results), but **re-shoot them after granting photo access on a fresh install**: the
|
||||
existing ones still show the "Photo access needed" banner — see the caveat in
|
||||
STORE_LISTING.md.
|
||||
⚠️ Play rejects a screenshot whose longer side is more than 2× its shorter side, so a
|
||||
1080×2400 phone capture has to be cropped to 1080×2160;
|
||||
`tools/make-store-screenshots.sh` does exactly that (status and navigation bars only).
|
||||
- [ ] Category: *Tools* (or *Photography*). Tags: photo cleanup, duplicates.
|
||||
|
||||
## 5. Policy and compliance
|
||||
|
||||
- [ ] Host [PRIVACY.md](../PRIVACY.md) at a public URL and paste that URL into the
|
||||
Play Console. **This is mandatory** because the app accesses photos.
|
||||
⚠️ **Before hosting:** replace the `support@example.com` placeholder in the
|
||||
*Contact* section with a real address — a policy whose contact is a placeholder is a
|
||||
rejection risk.
|
||||
|
||||
> **Note on Play's automatic translation:** it translates the *store listing*, not
|
||||
> the hosted policy. If you want the policy in another language, host a translated
|
||||
> copy yourself and link it from the listing.
|
||||
- [ ] Complete the **Data safety** form:
|
||||
- *Does your app collect or share any required user data types?* → **No**.
|
||||
- Data types accessed: **Photos** — processed on-device only, not collected.
|
||||
- *Is all user data encrypted in transit?* → N/A (no transmission).
|
||||
- *Do you provide a way for users to request data deletion?* → Uninstalling
|
||||
removes all app data.
|
||||
- [ ] Confirm the **permission set** the Console will show. Read it from the *merged*
|
||||
release manifest, not the source one, so nothing a library quietly adds slips past:
|
||||
|
||||
| Declared permission | Why | From |
|
||||
|---|---|---|
|
||||
| `READ_MEDIA_IMAGES` | read the photo library | app |
|
||||
| `READ_MEDIA_VISUAL_USER_SELECTED` | Android 14 partial access | app |
|
||||
| `READ_EXTERNAL_STORAGE` (`maxSdkVersion=32`) | same, on Android ≤ 12 | app |
|
||||
| `WRITE_EXTERNAL_STORAGE` (`maxSdkVersion=29`) | legacy delete on Android 10 only | app |
|
||||
| `FOREGROUND_SERVICE`, `FOREGROUND_SERVICE_DATA_SYNC` | keep a scan alive | app |
|
||||
| `POST_NOTIFICATIONS` | scan-progress notification | app |
|
||||
| `…DYNAMIC_RECEIVER_NOT_EXPORTED_PERMISSION` | framework-generated, not user-facing | AndroidX |
|
||||
|
||||
Deliberately **absent**: `INTERNET` and `MANAGE_EXTERNAL_STORAGE`. Both are
|
||||
load-bearing for the "nothing leaves the device" claim, so re-check after any new
|
||||
dependency:
|
||||
|
||||
```bash
|
||||
./gradlew :app:processReleaseMainManifest
|
||||
grep -A1 '<uses-permission' \
|
||||
app/build/intermediates/merged_manifests/release/processReleaseManifest/AndroidManifest.xml \
|
||||
| grep -oE 'android.permission.[A-Z_]+' | sort -u
|
||||
```
|
||||
|
||||
(Match `<uses-permission` only. A bare `android.permission.` grep also picks up
|
||||
`android.permission.DUMP`, which AndroidX puts on its profile-installer *receiver* —
|
||||
a component restriction, not a permission the app requests.)
|
||||
- [ ] Answer the **Photo and video permissions** declaration in the Console. The app
|
||||
requests `READ_MEDIA_IMAGES` (plus `READ_MEDIA_VISUAL_USER_SELECTED` for Android 14
|
||||
partial access) and deletes only user-selected files through the system dialog.
|
||||
Paste:
|
||||
|
||||
> Foto Dup Finder is a photo-cleanup tool. Reading the photo library **is** its core
|
||||
> functionality: it compares photos on the device to find exact and near duplicates
|
||||
> and lets the user remove the copies they select. Nothing is uploaded — the app
|
||||
> declares no network permission. Deletion always goes through the system consent
|
||||
> dialog (`MediaStore.createDeleteRequest`); the app never requests
|
||||
> `MANAGE_EXTERNAL_STORAGE`.
|
||||
|
||||
- [ ] Declare the **foreground service type**. This is the release-blocking item for the
|
||||
Android 14+ review. **[code]** The manifest already matches what you will declare:
|
||||
|
||||
| Where | Value |
|
||||
|---|---|
|
||||
| `AndroidManifest.xml` | `<uses-permission android:name="android.permission.FOREGROUND_SERVICE_DATA_SYNC" />` |
|
||||
| `AndroidManifest.xml` | `<service android:name=".service.ScanService" android:foregroundServiceType="dataSync" android:exported="false" />` |
|
||||
| `ScanService.kt` | `ServiceCompat.startForeground(…, ServiceInfo.FOREGROUND_SERVICE_TYPE_DATA_SYNC)` |
|
||||
| `ScanService.kt` | `onTimeout()` stops the scan at Android 15's 6-hour `dataSync` cap |
|
||||
|
||||
In the Console go to **App content › Foreground service permissions**, select
|
||||
**Data sync**, and paste:
|
||||
|
||||
> The app finds duplicate photos on the device. The user starts a scan and may leave
|
||||
> the app while it runs; the foreground service keeps that user-initiated scan alive
|
||||
> and shows its progress. The scan reads the photo library and writes a small local
|
||||
> index of file hashes. It performs no network access and transfers no data.
|
||||
|
||||
The declaration is **per type**: if a new foreground-service type is ever added to
|
||||
the manifest, it has to be declared here too.
|
||||
- [ ] Complete the **content rating** questionnaire (no objectionable content →
|
||||
typically rated Everyone / 3+).
|
||||
- [ ] Select the target audience (not designed for children).
|
||||
- [ ] Accept the *Developer Program Policies* and *US export laws* declarations.
|
||||
|
||||
## 6. Testing before production
|
||||
|
||||
- [ ] Create an **internal testing** track and add yourself as a tester; install via
|
||||
the Play link on a real device.
|
||||
- [ ] Test on the oldest supported Android version you can (minSdk 26) and on the
|
||||
newest you have (Android 16 / API 36).
|
||||
- [ ] Test the permission flows:
|
||||
- [ ] Deny access — app must not crash and must explain what is needed.
|
||||
- [ ] Android 14+ **partial access** ("Select photos") — banner appears and the
|
||||
app scans the subset.
|
||||
- [ ] Grant full access — scan works.
|
||||
- [ ] Test deletion:
|
||||
- [ ] Move to trash (Android 11+).
|
||||
- [ ] Permanent delete (Android 11+).
|
||||
- [ ] Refuse the system dialog — nothing is deleted, the app says so.
|
||||
- [ ] Test with a library of several thousand photos to sanity-check scan time.
|
||||
- [ ] Verify no crashes in *Play Console > Quality > Android vitals* after rollout.
|
||||
|
||||
## 7. Release notes ("What's new") and rollout
|
||||
|
||||
Release notes are **not stored in this repo** — they are typed into the Play Console
|
||||
for each release.
|
||||
|
||||
Where:
|
||||
|
||||
- Production: **Test and release > Production**, then **Create new release**
|
||||
- Testing: **Test and release > Testing > Internal / Closed / Open testing**
|
||||
|
||||
Rules (per Google's *Prepare and roll out a release* guide):
|
||||
|
||||
- Up to **500 Unicode characters per language**.
|
||||
- Language tags go on their own line, wrapping the text — a block starts with a line
|
||||
containing `<en-US>` and ends with a line containing `</en-US>`. Several
|
||||
`<lang>` blocks can be entered together.
|
||||
- Notes belong to a **release**, so every track has its own; "Copy from a previous
|
||||
release" avoids retyping.
|
||||
- Google explicitly forbids using release notes for **promotion or to solicit user
|
||||
actions** — keep them factual about what changed.
|
||||
- For app updates list only user-visible changes; the full developer history belongs
|
||||
in [CHANGELOG.md](../CHANGELOG.md).
|
||||
|
||||
Ready-to-paste block for the first release:
|
||||
|
||||
<en-US>
|
||||
Initial release.
|
||||
|
||||
• Finds exact and near-duplicate photos
|
||||
• Suggests which copy to keep
|
||||
• Moves photos to the trash or deletes them via the system dialog
|
||||
• No ads, no account, works fully offline
|
||||
</en-US>
|
||||
|
||||
Ready-to-paste block for **1.0.1** (the scan-robustness patch):
|
||||
|
||||
<en-US>
|
||||
• Fixed a scan that could appear to stop on very large photo libraries
|
||||
• The app now recovers gracefully when the photo library is temporarily unresponsive
|
||||
</en-US>
|
||||
|
||||
- [ ] Upload the `.aab`, paste the notes above, and roll out to **internal** →
|
||||
**closed** → **production** (staged rollout recommended).
|
||||
- [ ] Monitor Android vitals and reviews; respond to feedback.
|
||||
|
||||
## 8. Post-launch maintenance
|
||||
|
||||
- [ ] Play requires new apps/updates to target recent API levels (API 36 from
|
||||
August 2026 **[code]**). Re-check this requirement each year and bump
|
||||
`targetSdk`.
|
||||
- [ ] Watch for newer AGP; when it supports it, update `compileSdk` to 37 and bump
|
||||
the Compose BOM and Coil together (see the version note in the README).
|
||||
- [ ] Optionally add crash reporting later — but note this would change the Data
|
||||
safety answers, so weigh it against the "no data leaves the device" promise.
|
||||
|
||||
## 9. Optional monetisation
|
||||
|
||||
The app currently has no ads and no billing, which is a deliberate differentiator.
|
||||
If you later add a paid tier, use **Play Billing** (e.g. a one-time "Pro"
|
||||
purchase) and update:
|
||||
- the Data safety form (purchase history),
|
||||
- the privacy policy,
|
||||
- the store description.
|
||||
|
||||
Avoid ad SDKs entirely — they would contradict the app's main selling point.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Roadmap
|
||||
|
||||
Foto Dup Finder is built against a fixed milestone plan (**M0–M10**). This file is the
|
||||
single source of truth for what is done and what remains. Before this file existed the
|
||||
plan lived only in developer notes, and the [CHANGELOG](../CHANGELOG.md) refers to
|
||||
milestone labels (`M7`, `M8`).
|
||||
|
||||
Legend: ✅ done · 🚧 in progress · ⏳ not started · 💡 candidate (proposed, not scheduled)
|
||||
|
||||
## Milestones
|
||||
|
||||
| # | Milestone | Status | Evidence / notes |
|
||||
|---|---|---|---|
|
||||
| M0 | Toolchain & scaffold | ✅ | User-local SDK, Gradle 9.3.1 + AGP 9.1.0, wrapper and version catalog, empty debug APK builds |
|
||||
| M1 | Pure-Kotlin duplicate engine + JVM tests | ✅ | SHA-256 quick/full hashing, pHash + RGB dHash, union-find grouping, Exact/Balanced/Aggressive policies; tests under `app/src/test` |
|
||||
| M2 | Data layer | ✅ | MediaStore query, coroutine-parallel hashing, incremental SQLite hash index, permission + partial-access handling |
|
||||
| M3 | Compose UI | ✅ | Scan progress, grouped results with thumbnails, two-step staging, selection helpers |
|
||||
| M4 | Safe deletion | ✅ | `MediaStore.createDeleteRequest` / `createTrashRequest`; legacy direct delete on Android 10 |
|
||||
| M5 | Performance pass | ✅ | Size short-circuit, provider-thumbnail hashing, batched hashing (folded into M2/M3) |
|
||||
| M6 | Release config & Play readiness | ✅ | R8/minify, optional signing, `.aab`, [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md), [PRIVACY.md](../PRIVACY.md), README |
|
||||
| M7 | On-device validation & hardening | ✅ | Foreground `dataSync` service, one failure boundary per file, `core/DecodeMath`, hostile fixtures ([EDGE_CASE_TESTING](EDGE_CASE_TESTING.md)), `versionName` → 1.0.0 |
|
||||
| M8 | Localisation groundwork | ✅ | Every user-visible string in `res/values/strings.xml` with real `<plurals>`, shared `text/TextRes.kt`; **English is the only locale** |
|
||||
| M9 | Monetisation | ⏳ | No billing code. Blocked on a product decision (free forever vs. one-time unlock vs. limited free scans); changes the Data Safety answers |
|
||||
| M10 | Store assets | ✅ | `tools/make-store-assets.sh` renders the 512×512 icon and the 1024×500 feature graphic from the app's own vectors; `tools/make-store-screenshots.sh` normalises device captures to Play's 2:1 limit (1080×2160); all listing copy is drafted and length-checked in [STORE_LISTING](STORE_LISTING.md). The captured screenshots still show the permission banner — see blockers |
|
||||
|
||||
## Release blockers (work that is not a numbered milestone)
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| ~~Large-library scan stall~~ ✅ **fixed — verified on device** | The 5,363-photo library used to freeze at *“74 / 100 · Comparing file signatures…”* (a **blocked syscall**: zero CPU, workers sleeping). `BoundedCall` now abandons a stuck provider call **and** trips a stall breaker after three consecutive timeouts. **Verified on the real test device (Android 12 / MIUI): the scan completes — 20.2 s cold, 11.0 s warm, 15 groups, 6 unreadable — where it used to hang.** Covered by `BoundedCallTest` and `ScanEngineEndToEndTest` |
|
||||
| ~~Unreleased `BoundedCall` fix~~ ✅ | The provider-call deadline shipped in **1.0.1** (`versionCode` 2) |
|
||||
| **Play Console: foreground-service declaration** | Declaring the `dataSync` type (App content › Foreground service permissions) is release-blocking. **[code]** the manifest already declares it; the ready-to-paste justification is in [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) step 5 — this is a Console action, not a code change |
|
||||
| **Remaining Play paperwork** | Keystore / Play App Signing, Data Safety form, hosted privacy policy and store listing — see [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) |
|
||||
| **Store screenshots still show the permission banner** | The three captures were taken with the debug-only `MANAGE_EXTERNAL_STORAGE` app-op, so the app's own permission check (correctly) still reported *no access* and the banner is in frame. Re-shoot on a clean install after tapping **Grant access** — the same run that closes the permission-flow verification gap. Files, specs and caveats: [STORE_LISTING](STORE_LISTING.md) |
|
||||
| **Device-only verifications** | RAW/DNG files, a *real* iPhone HEIC with a non-upright EXIF orientation, permission revoked mid-scan, and the Android 14 “Selected photos” flow. ✅ The **index-reuse** check is done: after a scan the index holds **5,363 rows, 5,357 with a perceptual hash**, and a second scan runs in **11.0 s vs 20.2 s** cold |
|
||||
|
||||
## Candidate milestones (proposed, not scheduled)
|
||||
|
||||
| Candidate | Rationale |
|
||||
|---|---|
|
||||
| **Colour signature** | Flat-colour images (blank scans, dark UI screenshots) hash identically regardless of colour, so they are reported as mutual duplicates — see the “Known gap” in [EDGE_CASE_TESTING](EDGE_CASE_TESTING.md). A small colour histogram is orthogonal to the grayscale structure hash, and the benchmark harness already has an implementation to measure against |
|
||||
| **Video support** | Only images are scanned today (README “Known limitations”) |
|
||||
| **“Similar shots” section** | Bursts / same-subject shots as a separate, never-pre-selected list. Needs a different algorithm and must not feed the deletion flow |
|
||||
| **Warm-scan speed** | A second scan is faster (11.0 s) but its “similar” stage still costs ~10 s even though all 5,357 perceptual hashes are cached — the per-photo `withTimeoutOrNull` wrapper still runs for all 5,363 photos. Profiling this could make an unchanged library re-scan in well under a second |
|
||||
|
||||
## Suggested order
|
||||
|
||||
1. ✅ **Done** — the scan-stall hardening and the `BoundedCall` fix shipped as **1.0.1**
|
||||
(`versionCode` 2). Only the on-device re-check of the 5,363-photo scan remains.
|
||||
2. Finish the Play Console paperwork and roll out through **internal → closed → production**.
|
||||
3. ✅ **Done** — **M10 store assets**: the icon and feature graphic are generated from the
|
||||
app's own vectors, the screenshot normaliser enforces Play's 2:1 limit, and the listing
|
||||
copy is drafted and length-checked. The only remainder is the screenshot re-shoot after a
|
||||
real permission grant (see blockers).
|
||||
4. **Colour signature** — the flat-colour false-positive fix.
|
||||
5. **M9 — monetisation**, once the product decision is made.
|
||||
|
||||
Deliberately deferred: **video support** and the **“similar shots”** section.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Play Store listing (M10)
|
||||
|
||||
Everything the *Main store listing* page in the Play Console asks for, ready to copy. The
|
||||
graphics are generated from the app's own vectors, so they can be regenerated after any
|
||||
branding change instead of being edited by hand.
|
||||
|
||||
See [PLAY_RELEASE_CHECKLIST](PLAY_RELEASE_CHECKLIST.md) § 4 for where these fit in the
|
||||
release.
|
||||
|
||||
## Graphics
|
||||
|
||||
Generated into `store-assets/` **inside the repository**, never into `$HOME`. These PNGs are
|
||||
the files that get uploaded to the Play Console, so they are committed and versioned with the
|
||||
release they belong to. Only the uncropped captures the screenshots are cut from
|
||||
(`store-assets/raw-screenshots/`) are git-ignored.
|
||||
|
||||
```bash
|
||||
./tools/make-store-assets.sh # icon + feature graphic -> store-assets/
|
||||
./tools/make-store-screenshots.sh # crop the adb captures -> store-assets/screenshots/
|
||||
```
|
||||
|
||||
| Asset | Play's spec | Result | Source |
|
||||
|---|---|---|---|
|
||||
| App icon | 512×512, 32-bit PNG with alpha, ≤ 1 MB | `store-assets/icon-512.png` — 512×512, RGBA, 6 KB | `tools/store-assets/icon.svg`, mirrored from `app/src/main/res/drawable/ic_launcher_background.xml` + `ic_launcher_foreground.xml` (paths and group transforms are identical, so the store icon *is* the launcher icon) |
|
||||
| Feature graphic | 1024×500, JPEG or 24-bit PNG without alpha | `store-assets/feature-graphic-1024x500.png` — 1024×500, RGB | `tools/store-assets/feature-graphic.svg` |
|
||||
| Phone screenshots | JPEG or 24-bit PNG without alpha, shorter side ≥ 320 px, longer side ≤ 3840 px, **longer side ≤ 2× shorter side** | `store-assets/screenshots/0{1,2,3}-*.png` — 1080×2160, RGB | `tools/make-store-screenshots.sh`, from raw `adb exec-out screencap` captures in `store-assets/raw-screenshots/` |
|
||||
|
||||
Two details worth keeping in mind:
|
||||
|
||||
- The icon is **full bleed**: the adaptive-icon background covers all 108 dp, so every pixel
|
||||
of the PNG ends up opaque and Play's mask can crop to any shape without showing a gap.
|
||||
- A phone screenshot at the test device's native 1080×2400 is **2.22:1 and therefore
|
||||
rejected** — Play's limit is 2:1. The normaliser crops the status bar (93 px) and the
|
||||
navigation bar (130 px) and clamps to 1080×2160, which removes system chrome only, never
|
||||
app content. Re-measure both insets if you shoot on another device and pass
|
||||
`STATUS_BAR_PX` / `NAV_BAR_PX`.
|
||||
|
||||
## Screenshots
|
||||
|
||||
Captured on the test device (1080×2400, Android 12 / MIUI), cropped to 1080×2160:
|
||||
|
||||
| File | Screen |
|
||||
|---|---|
|
||||
| `01-empty.png` | Intro / empty state before the first scan |
|
||||
| `02-scanning.png` | Scanning progress — *“Looking for visually similar photos…”* 5341 / 5363 |
|
||||
| `03-results.png` | Results — 15 duplicate groups, 39.8 MB reclaimable, `BEST` badge, selection bar |
|
||||
|
||||
⚠️ **Two caveats before these go live.**
|
||||
|
||||
1. All three still show the *“Photo access needed”* banner. The test device (Xiaomi M2101K7AG
|
||||
/ MIUI) refuses `pm grant`, `install -g` and synthetic taps, so the runtime permission was
|
||||
never actually granted there — the scan was driven with a debug-only
|
||||
`MANAGE_EXTERNAL_STORAGE` app-op, which the app's permission check does not honour (see
|
||||
[EDGE_CASE_TESTING](EDGE_CASE_TESTING.md)). **Re-shoot on a clean install after tapping
|
||||
“Grant access”**: that also closes the one remaining device-only verification gap.
|
||||
2. The results screenshot shows real photos from the test library. They are the developer's
|
||||
own, so they can be published, but check that nothing personal is recognisable first.
|
||||
|
||||
Screens still worth capturing once a device grants access through the real flow — each needs
|
||||
one tap, which is exactly what the MIUI test device cannot be driven to do:
|
||||
|
||||
- the **Options** sheet with the three sensitivity presets,
|
||||
- the staged state (*“All duplicates are ready to go”*) after selecting duplicates,
|
||||
- Android's **confirmation dialog** for moving photos to the trash.
|
||||
|
||||
## Store listing copy
|
||||
|
||||
### App name (max 30 characters)
|
||||
|
||||
Foto Dup Finder
|
||||
|
||||
### Short description (max 80 characters)
|
||||
|
||||
Find and remove duplicate photos. Free, ad-free, works offline.
|
||||
|
||||
Alternative, if the no-upload promise should lead:
|
||||
|
||||
Find and delete duplicate photos. No ads, no uploads, no account.
|
||||
|
||||
### Full description (max 4000 characters)
|
||||
|
||||
Find and remove duplicate photos, then get the space back. Foto Dup Finder does one
|
||||
job and does it without ads, without an account and without ever uploading anything.
|
||||
|
||||
WHY IT IS DIFFERENT
|
||||
• No ads — no banners, no interstitials, no upsells.
|
||||
• No internet permission at all, so the app has no way to upload your photos.
|
||||
• No account, no sign-in, no tracking, no analytics.
|
||||
|
||||
WHAT IT DOES
|
||||
• Finds exact copies: files that are byte-for-byte identical, even when their names
|
||||
differ.
|
||||
• Finds near-duplicates: the same picture saved twice, or two shots of the same
|
||||
scene. A perceptual hash compares how the images actually look rather than only
|
||||
their bytes, so a re-compressed or slightly re-cropped copy is still caught.
|
||||
• Suggests what to keep. The best copy of each group is marked BEST, and
|
||||
"Select all but best" marks the rest in a single tap.
|
||||
• Lets you choose how far to go: Exact copies only, Balanced (recommended) or
|
||||
Aggressive.
|
||||
• Deletes safely. Every removal goes through Android's own confirmation dialog, and
|
||||
you choose between moving photos to the trash (recoverable) and deleting them
|
||||
permanently. Nothing is removed without your confirmation.
|
||||
• Copes with a messy library. RAW files and unreadable or corrupt images are skipped
|
||||
and reported instead of stopping the scan.
|
||||
• Much faster the second time. File hashes are remembered on the device, so
|
||||
re-scanning an unchanged library takes a fraction of the first run.
|
||||
|
||||
HOW IT WORKS
|
||||
1. Grant photo access — everything runs on your device.
|
||||
2. Tap Start scan. A library of several thousand photos is done in seconds.
|
||||
3. Review the groups, keep the copies you want and select the rest.
|
||||
4. Tap "Remove all selected" and confirm once for the whole batch.
|
||||
|
||||
YOUR PHOTOS STAY YOURS
|
||||
Foto Dup Finder declares no internet permission, ships no advertising or analytics
|
||||
SDKs, and stores nothing besides a local index of file hashes so the next scan can be
|
||||
quick. No photo ever leaves your phone, and uninstalling the app removes the index
|
||||
with it.
|
||||
|
||||
GOOD TO KNOW
|
||||
• Requires Android 8.0 or newer.
|
||||
• Photos moved to the trash can be restored from your gallery app until the trash is
|
||||
emptied (Android 11 and newer).
|
||||
• Works fully offline, in flight mode, always.
|
||||
|
||||
Every claim above is checkable against this repository: no `INTERNET` permission in the
|
||||
merged manifest, no ad or analytics dependency in `gradle/libs.versions.toml`, `minSdk 26`
|
||||
in `app/build.gradle.kts`, and the second-run speed-up is the persistent hash index in
|
||||
`data/PhotoIndexStore.kt`.
|
||||
|
||||
### Category and tags
|
||||
|
||||
- **Category:** Tools (alternative: Photography — *Tools* matches the one-job utility framing
|
||||
better, and Photography is more crowded).
|
||||
- **Tags:** duplicate photos, photo cleanup, storage.
|
||||
- **Contact details:** the same e-mail as in [PRIVACY.md](../PRIVACY.md) — replace the
|
||||
`support@example.com` placeholder there first.
|
||||
Reference in New Issue
Block a user