SoftwareApp
f45e3b79ab
Two Console-facing subjects, both decided here instead of in the Console. Tablet screenshots (7-inch and 10-inch) -------------------------------------- Play keeps a separate screenshot slot per form factor and asks for at least four screenshots on tablets, so the four phone captures did not cover them. Both sets are now captured on AOSP Android 12 AVDs, over the same generated 22-photo demo library, and normalised with FORM_FACTOR=tablet-7in|tablet-10in: - 7-inch - Nexus 7 2013: 1200x1920 portrait, sw600dp, 320 dpi, four files - 10-inch - Medium Tablet: 1600x2560 portrait, sw800dp, 320 dpi, four files This machine has no /dev/kvm - the node cannot even be created, because mknod is refused for the container as root - so the emulator ran in pure software emulation (-accel off -no-window -gpu swiftshader_indirect): about 3 minutes to boot, 1-12 s per screenshot. hide_error_dialogs was switched on and the animation scales off, because under TCG SystemUI raises an ANR dialog that would otherwise sit in the frame; the 10-inch profile boots landscape and its portrait rotation is 3, not 0. The scanning frame comes from a screencap burst started before the tap, since 22 photos scan in seconds, and the tap coordinates come from uiautomator dumps rather than guesswork. The results screens are not staged: a finished scan auto-selects all but best, so "14 selected, 43.7 MB" is the default state. 2 duplicate groups, 16 of 22 photos, 43.7 MB reclaimable, scanned in 39.7 s (7-inch) and 53.1 s (10-inch). Nothing except the demo library was ever pushed to either AVD and it is deleted again afterwards, so no real photo can appear in the tablet sets. Screenshots stay out of git, as before. App pricing: a one-time paid app -------------------------------- The one irreversible decision in this release. Play's rule: an app can go paid -> free, but "once your app has been offered for free, the app can't be changed to paid" - a new package name would be needed - so this had to be settled before the first upload, which has not happened yet (versionCode 2). Chosen over free with a Pro unlock because it is the only model that costs the app nothing: no Billing dependency, no com.android.vending.BILLING permission, no Data Safety answers, and the "no internet permission at all" claim survives untouched. Checked rather than assumed: unpacking com.android.billingclient:billing:9.1.0 shows its manifest declares only com.android.vending.BILLING and the billing service queries, so in-app purchase would not have added INTERNET either. Billing 8+ is the current requirement (latest 9.1.0) if that is ever revisited. Price: base EUR 2.99 with a 14-day 50% launch sale at EUR 1.49. A EUR 1 base price was considered and rejected: a sale can always be added, but a low anchor cannot be undone, and a paid app's first transaction is its only transaction. Play's fee for a paid app's initial purchase is 10% + 5% billing fee (~15%) for the first $1M/year of new-install transactions in the EEA, so EUR 2.99 nets ~EUR 2.54 against ~EUR 0.85 at EUR 1. The sale rules shape the launch: 1-14 days live, 30 days between one sale ending and the next starting, and the app must be published before a sale can be created. The strikethrough does appear in Germany, which is not one of the 12 EU states and the UK that hide it. The listing copy had to stop saying "free": the short description is now "Find and remove duplicate photos. Ad-free, works offline, no uploads." (69/80 chars) and the full description gains one factual pay-once line plus a "sold through Google Play as a one-time purchase" bullet (2,426/4,000). The price stays out of the title, icon and developer name, because Play's metadata policy forbids price and promotional wording there. PRIVACY.md gains a Purchases section - Google Play processes the payment, the app has no payment code and no network access at all - and its date moves to 2026-10-14, so the published copy needs re-pushing to Gitea. No code changed, so the signed 1.0.1 .aab is untouched (sha256 f8e6a29a...). What is left is Console work: a payments profile, the price and the launch sale on Products > App pricing, and a promo-code copy to test the paid listing end to end.
234 lines
10 KiB
Markdown
234 lines
10 KiB
Markdown
# Foto Dup Finder
|
||
|
||
A fast, **ad-free**, **fully offline** Android app that finds and removes duplicate
|
||
and near-duplicate photos. Built as a direct answer to the slow, ad-laden cleanup
|
||
tools that ship with many phones.
|
||
|
||
Nothing ever leaves the device: the app declares no network permission, contains no
|
||
analytics and needs no account.
|
||
|
||
## Features
|
||
|
||
- **Two kinds of duplicates**
|
||
- *Exact* — byte-identical copies, found with a cheap size/head/tail signature
|
||
followed by a full SHA-256.
|
||
- *Near* — resized, recompressed or lightly edited versions, found with a 64-bit
|
||
perceptual difference hash (`dHash`) plus a Hamming-distance threshold.
|
||
- **Tiered, parallel pipeline**: only plausible candidates get a full hash, and the
|
||
work is spread across all CPU cores.
|
||
- **Persistent index** so a re-scan only touches new or changed files.
|
||
- **Safe deletion** through the system consent dialog
|
||
(`MediaStore.createDeleteRequest`) or the system trash (`createTrashRequest`).
|
||
The app never requests `MANAGE_EXTERNAL_STORAGE`.
|
||
- **Two-step cleanup**: removing a row only *stages* it (the row disappears instantly,
|
||
nothing is deleted); one **Apply** then performs the removal, so Android asks for a
|
||
single confirmation for the whole batch instead of one per group.
|
||
- **Sensible defaults**: “select all but best” keeps the largest (then newest) copy
|
||
of each group.
|
||
- **Android 14+ partial photo access** handled explicitly, with a clear prompt to
|
||
upgrade to full access.
|
||
|
||
## Tech stack
|
||
|
||
| Area | Choice |
|
||
|---|---|
|
||
| Language | Kotlin 2.4.10 (using AGP 9 *built-in Kotlin*) |
|
||
| UI | Jetpack Compose, BOM 2026.06.01, Material 3 |
|
||
| Build | Gradle 9.3.1, Android Gradle Plugin 9.1.0 |
|
||
| Image loading | Coil 3.5.0 |
|
||
| Storage | MediaStore read + a small SQLite hash cache |
|
||
| SDK levels | `compileSdk`/`targetSdk` 36, `minSdk` 26 |
|
||
|
||
`targetSdk` is 36 because Google Play requires new apps and updates to target
|
||
Android 16 (API 36) from 31 August 2026.
|
||
|
||
> **Version note:** the Compose BOM is pinned to 2026.06.01 (Compose 1.11.4) and
|
||
> Coil to 3.5.0, because the next releases require `compileSdk 37`, which AGP 9.1.0
|
||
> does not support yet. Bump both together once a newer AGP lands.
|
||
|
||
## Project layout
|
||
|
||
```
|
||
app/src/main/java/de/softwareapp_hb/fotodup/finder/
|
||
├── core/ Pure-Kotlin duplicate engine (no Android imports!)
|
||
│ ├── Hashing.kt SHA-256 + cheap size/head/tail signature
|
||
│ ├── PerceptualHash.kt 64-bit dHash + Hamming distance
|
||
│ ├── DuplicateGrouper.kt exact + perceptual grouping (union-find)
|
||
│ ├── DuplicatePolicy.kt Exact / Balanced / Aggressive thresholds
|
||
│ └── …
|
||
├── data/ MediaStore access, hashing, permissions, SQLite index, deletion
|
||
├── domain/ ScanEngine (tiered pipeline) + progress/result models
|
||
└── ui/ Compose theme, scanner screen, ViewModel, components
|
||
app/src/test/java/… JVM unit tests for the core engine
|
||
```
|
||
|
||
`core/` deliberately has **no Android dependencies**, so the trickiest logic
|
||
(hashing, perceptual hashing, grouping) is covered by fast JVM tests.
|
||
|
||
## Building
|
||
|
||
Prerequisites:
|
||
|
||
- JDK 17+ (JDK 25 verified)
|
||
- Android SDK with `platforms;android-36` and `build-tools;36.0.0`
|
||
|
||
```bash
|
||
# Tell Gradle where the SDK lives (this file is git-ignored)
|
||
echo "sdk.dir=$HOME/Android/Sdk" > local.properties
|
||
|
||
./gradlew :app:assembleDebug # debug APK
|
||
./gradlew :app:testDebugUnitTest # JVM unit tests for the engine
|
||
./gradlew :app:lintDebug # Android Lint
|
||
./gradlew :app:assembleRelease # R8-minified release APK
|
||
./gradlew :app:bundleRelease # Play-ready .aab
|
||
```
|
||
|
||
### Installing on a device
|
||
|
||
```bash
|
||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||
```
|
||
|
||
## How detection works
|
||
|
||
```
|
||
MediaStore query
|
||
│
|
||
▼
|
||
1. quick signature = SHA-256(size + first & last 32 KB)
|
||
│ bucket by (size, signature)
|
||
▼
|
||
2. full SHA-256 only for buckets with 2+ members → EXACT duplicates
|
||
│
|
||
▼
|
||
3. perceptual fingerprint, both from ONE decode → NEAR duplicates
|
||
· pHash: 32x32 grayscale → DCT → 8x8 low-frequency block → 64 bits
|
||
· RGB dHash: one 64-bit difference hash per colour channel
|
||
│ candidates via 8-bit band index; a pair must pass BOTH tests
|
||
│ the pHash cut is fitted per scan (see below), then union-find clusters
|
||
4. SQLite hash cache → the next scan only re-reads changed files
|
||
```
|
||
|
||
**Why two descriptors?** pHash sees luminance structure, the RGB hashes see colour
|
||
structure. A single Hamming threshold cannot hold out as a library grows, because
|
||
unrelated pairs accumulate with the square of the library size. Measured on the
|
||
benchmark harness, the widest pHash threshold with *zero* false pairs fell from 13
|
||
(120 images) to 5 (2 880 images) — so the app fits the cut to the library instead of
|
||
trusting a constant, and requires the colour structure to agree as well.
|
||
|
||
Sensitivity presets (both values are ceilings; the pHash one is adapted per scan):
|
||
|
||
| Policy | pHash ceiling | RGB ceiling | Notes |
|
||
|---|---|---|---|
|
||
| Exact copies only | – | – | byte-identical files, no decoding at all |
|
||
| Balanced (default) | 12 bits | 60 / 192 | the recommended setting |
|
||
| Aggressive | 18 bits | 84 / 192 | catches heavier re-edits |
|
||
|
||
Banding gives *exact* candidate recall for pHash distances up to 7 (see the comment in
|
||
`DuplicateGrouper`).
|
||
|
||
## Permissions
|
||
|
||
| Permission | Why it is needed |
|
||
|---|---|
|
||
| `READ_MEDIA_IMAGES` (API 33+) | read the photo library |
|
||
| `READ_EXTERNAL_STORAGE` (API ≤ 32) | same, on older devices |
|
||
| `READ_MEDIA_VISUAL_USER_SELECTED` (API 34+) | Android 14 "Selected photos" mode |
|
||
| `WRITE_EXTERNAL_STORAGE` (**API ≤ 29 only**) | legacy delete path on Android 10 |
|
||
| `POST_NOTIFICATIONS` (API 33+) | shows the scan-progress notification (optional — the scan runs without it) |
|
||
| `FOREGROUND_SERVICE` + `FOREGROUND_SERVICE_DATA_SYNC` | keeps a long scan alive while the app is backgrounded |
|
||
|
||
Deletion on Android 11+ always goes through the system consent dialog. The app
|
||
never requests `MANAGE_EXTERNAL_STORAGE`.
|
||
|
||
### Scanning in the background
|
||
|
||
A scan is owned by `ScanController` (process-scoped, created by the Application), not
|
||
by the screen's ViewModel. While one is running, `ScanService` holds a `dataSync`
|
||
foreground service with a progress notification that has a **Cancel** action. So:
|
||
|
||
- leaving the app does not kill the scan,
|
||
- the result survives screen rotation and re-entry,
|
||
- the user always has a visible way to stop it.
|
||
|
||
When `POST_NOTIFICATIONS` is denied the scan still works; the notification is simply
|
||
not shown.
|
||
|
||
## Testing
|
||
|
||
`app/src/test` holds JVM unit tests for `core/`: SHA-256 test vectors, dHash bit
|
||
patterns, Hamming distance, exact/perceptual grouping, aspect-ratio filtering and
|
||
the reclaimable-bytes maths.
|
||
|
||
```bash
|
||
./gradlew :app:testDebugUnitTest
|
||
```
|
||
|
||
There is no instrumentation test, so UI behaviour is verified by installing the debug build on a
|
||
device or an emulator (`adb install -r app/build/outputs/apk/debug/app-debug.apk`) and driving it by
|
||
hand or with `adb shell input tap`. That is how the store screenshots are taken; the tablet sets were
|
||
shot on AOSP AVDs running in **software emulation**, because this machine exposes no `/dev/kvm`
|
||
(see [docs/STORE_LISTING.md](docs/STORE_LISTING.md)).
|
||
|
||
## Releasing to Google Play
|
||
|
||
See [docs/PLAY_RELEASE_CHECKLIST.md](docs/PLAY_RELEASE_CHECKLIST.md) (signing, store
|
||
listing, Data Safety, release notes, rollout), [docs/STORE_LISTING.md](docs/STORE_LISTING.md)
|
||
(ready-to-paste listing copy plus the icon / feature graphic generators) and
|
||
[PRIVACY.md](PRIVACY.md).
|
||
|
||
The app is published as a **one-time paid app** — no ads, no subscription, no in-app purchases, and
|
||
therefore no Billing code or `com.android.vending.BILLING` permission in the APK: the price lives on
|
||
the listing, not in the build ([why](docs/STORE_LISTING.md#app-pricing)).
|
||
|
||
The upload-ready graphics and screenshots live in [`store-assets/`](store-assets/).
|
||
`tools/make-store-assets.sh` and `tools/make-store-screenshots.sh` regenerate them from the
|
||
app's own icon vectors and from `adb` captures — always inside the repo, never in `$HOME`.
|
||
Screenshots are kept per form factor (`store-assets/{raw-,}screenshots/{phone,tablet-7in,tablet-10in}/`)
|
||
because Play gives phones and each tablet size a slot of their own: `FORM_FACTOR=tablet-7in`
|
||
selects the 7-inch pipeline, and the tablet runs deliberately crop no system chrome because a
|
||
tablet frame is already inside Play's 2:1 cap. All three sets are captured — the phone set on a real
|
||
device, the tablet sets on AOSP AVDs; see [docs/STORE_LISTING.md](docs/STORE_LISTING.md).
|
||
`python3 tools/make-demo-photos.py` draws the synthetic photo library the results screenshot is
|
||
captured over, so the listing never shows a real one.
|
||
|
||
User-visible changes are tracked in [CHANGELOG.md](CHANGELOG.md); the Play "What's new"
|
||
text you type per release is separate and lives only in the Play Console.
|
||
|
||
## Localisation
|
||
|
||
All user-visible text lives in `app/src/main/res/values/strings.xml`, with proper `<plurals>`
|
||
where a count can be one, and both the screen and the progress notification resolve their
|
||
labels through `text/TextRes.kt`. **English is currently the only locale.**
|
||
|
||
Adding a language is then a file, not a refactor:
|
||
|
||
```
|
||
app/src/main/res/values-de/strings.xml # same keys, translated values
|
||
```
|
||
|
||
One caveat worth knowing: **Google Play's automatic translation translates your store
|
||
listing, not the app UI.** Translated UI text has to be supplied in `values-<locale>/`, by a
|
||
translator or by running the strings file through a translation service and reviewing it.
|
||
|
||
Byte units ("KB", "MB") and the duration formatter are intentionally left unlocalised.
|
||
|
||
## Known limitations
|
||
|
||
- Only **images** are scanned — videos are not supported yet.
|
||
- Near-duplicate detection ignores EXIF rotation, so an image saved physically
|
||
rotated by 90° will not match its original.
|
||
- The Aggressive policy may miss some pairs because of the banded candidate search.
|
||
- The very first scan of a huge library (50k+ photos) takes a while; later scans
|
||
reuse the hash index and are much faster.
|
||
|
||
## Roadmap
|
||
|
||
The project is tracked against a milestone plan (M0–M10). What is done, what remains,
|
||
the current release blockers and the candidate features all live in
|
||
[docs/ROADMAP.md](docs/ROADMAP.md).
|
||
|
||
## Licence
|
||
|
||
Proprietary — all rights reserved. Change this if you intend to open-source it.
|