Initial commit: Foto Dup Finder 1.0.1
An Android duplicate-photo finder. Pure-Kotlin core (SHA-256 + dHash hashing, exact and perceptual union-find grouping, three sensitivity presets) with MediaStore access, a persistent SQLite hash index and a tiered ScanEngine behind a Jetpack Compose UI. No INTERNET permission, no ads, no analytics, no account. Ships versionCode 2 / versionName 1.0.1, carrying the scan-stall hardening (BoundedCall breaker) and verified against a 5,363-photo device library. 80 JVM unit tests, lint clean. Release paperwork is in docs/ (Play checklist, store listing, edge-case testing), the upload-ready Play graphics and screenshots in store-assets/, and the generators for those assets in tools/.
This commit is contained in:
Hadrian Burkhardt
@@ -0,0 +1,219 @@
|
||||
# Foto Dup Finder
|
||||
|
||||
A fast, **ad-free**, **fully offline** Android app that finds and removes duplicate
|
||||
and near-duplicate photos. Built as a direct answer to the slow, ad-laden cleanup
|
||||
tools that ship with many phones.
|
||||
|
||||
Nothing ever leaves the device: the app declares no network permission, contains no
|
||||
analytics and needs no account.
|
||||
|
||||
## Features
|
||||
|
||||
- **Two kinds of duplicates**
|
||||
- *Exact* — byte-identical copies, found with a cheap size/head/tail signature
|
||||
followed by a full SHA-256.
|
||||
- *Near* — resized, recompressed or lightly edited versions, found with a 64-bit
|
||||
perceptual difference hash (`dHash`) plus a Hamming-distance threshold.
|
||||
- **Tiered, parallel pipeline**: only plausible candidates get a full hash, and the
|
||||
work is spread across all CPU cores.
|
||||
- **Persistent index** so a re-scan only touches new or changed files.
|
||||
- **Safe deletion** through the system consent dialog
|
||||
(`MediaStore.createDeleteRequest`) or the system trash (`createTrashRequest`).
|
||||
The app never requests `MANAGE_EXTERNAL_STORAGE`.
|
||||
- **Two-step cleanup**: removing a row only *stages* it (the row disappears instantly,
|
||||
nothing is deleted); one **Apply** then performs the removal, so Android asks for a
|
||||
single confirmation for the whole batch instead of one per group.
|
||||
- **Sensible defaults**: “select all but best” keeps the largest (then newest) copy
|
||||
of each group.
|
||||
- **Android 14+ partial photo access** handled explicitly, with a clear prompt to
|
||||
upgrade to full access.
|
||||
|
||||
## Tech stack
|
||||
|
||||
| Area | Choice |
|
||||
|---|---|
|
||||
| Language | Kotlin 2.4.10 (using AGP 9 *built-in Kotlin*) |
|
||||
| UI | Jetpack Compose, BOM 2026.06.01, Material 3 |
|
||||
| Build | Gradle 9.3.1, Android Gradle Plugin 9.1.0 |
|
||||
| Image loading | Coil 3.5.0 |
|
||||
| Storage | MediaStore read + a small SQLite hash cache |
|
||||
| SDK levels | `compileSdk`/`targetSdk` 36, `minSdk` 26 |
|
||||
|
||||
`targetSdk` is 36 because Google Play requires new apps and updates to target
|
||||
Android 16 (API 36) from 31 August 2026.
|
||||
|
||||
> **Version note:** the Compose BOM is pinned to 2026.06.01 (Compose 1.11.4) and
|
||||
> Coil to 3.5.0, because the next releases require `compileSdk 37`, which AGP 9.1.0
|
||||
> does not support yet. Bump both together once a newer AGP lands.
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
app/src/main/java/de/softwareapp_hb/fotodup/finder/
|
||||
├── core/ Pure-Kotlin duplicate engine (no Android imports!)
|
||||
│ ├── Hashing.kt SHA-256 + cheap size/head/tail signature
|
||||
│ ├── PerceptualHash.kt 64-bit dHash + Hamming distance
|
||||
│ ├── DuplicateGrouper.kt exact + perceptual grouping (union-find)
|
||||
│ ├── DuplicatePolicy.kt Exact / Balanced / Aggressive thresholds
|
||||
│ └── …
|
||||
├── data/ MediaStore access, hashing, permissions, SQLite index, deletion
|
||||
├── domain/ ScanEngine (tiered pipeline) + progress/result models
|
||||
└── ui/ Compose theme, scanner screen, ViewModel, components
|
||||
app/src/test/java/… JVM unit tests for the core engine
|
||||
```
|
||||
|
||||
`core/` deliberately has **no Android dependencies**, so the trickiest logic
|
||||
(hashing, perceptual hashing, grouping) is covered by fast JVM tests.
|
||||
|
||||
## Building
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- JDK 17+ (JDK 25 verified)
|
||||
- Android SDK with `platforms;android-36` and `build-tools;36.0.0`
|
||||
|
||||
```bash
|
||||
# Tell Gradle where the SDK lives (this file is git-ignored)
|
||||
echo "sdk.dir=$HOME/Android/Sdk" > local.properties
|
||||
|
||||
./gradlew :app:assembleDebug # debug APK
|
||||
./gradlew :app:testDebugUnitTest # JVM unit tests for the engine
|
||||
./gradlew :app:lintDebug # Android Lint
|
||||
./gradlew :app:assembleRelease # R8-minified release APK
|
||||
./gradlew :app:bundleRelease # Play-ready .aab
|
||||
```
|
||||
|
||||
### Installing on a device
|
||||
|
||||
```bash
|
||||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||||
```
|
||||
|
||||
## How detection works
|
||||
|
||||
```
|
||||
MediaStore query
|
||||
│
|
||||
▼
|
||||
1. quick signature = SHA-256(size + first & last 32 KB)
|
||||
│ bucket by (size, signature)
|
||||
▼
|
||||
2. full SHA-256 only for buckets with 2+ members → EXACT duplicates
|
||||
│
|
||||
▼
|
||||
3. perceptual fingerprint, both from ONE decode → NEAR duplicates
|
||||
· pHash: 32x32 grayscale → DCT → 8x8 low-frequency block → 64 bits
|
||||
· RGB dHash: one 64-bit difference hash per colour channel
|
||||
│ candidates via 8-bit band index; a pair must pass BOTH tests
|
||||
│ the pHash cut is fitted per scan (see below), then union-find clusters
|
||||
4. SQLite hash cache → the next scan only re-reads changed files
|
||||
```
|
||||
|
||||
**Why two descriptors?** pHash sees luminance structure, the RGB hashes see colour
|
||||
structure. A single Hamming threshold cannot hold out as a library grows, because
|
||||
unrelated pairs accumulate with the square of the library size. Measured on the
|
||||
benchmark harness, the widest pHash threshold with *zero* false pairs fell from 13
|
||||
(120 images) to 5 (2 880 images) — so the app fits the cut to the library instead of
|
||||
trusting a constant, and requires the colour structure to agree as well.
|
||||
|
||||
Sensitivity presets (both values are ceilings; the pHash one is adapted per scan):
|
||||
|
||||
| Policy | pHash ceiling | RGB ceiling | Notes |
|
||||
|---|---|---|---|
|
||||
| Exact copies only | – | – | byte-identical files, no decoding at all |
|
||||
| Balanced (default) | 12 bits | 60 / 192 | the recommended setting |
|
||||
| Aggressive | 18 bits | 84 / 192 | catches heavier re-edits |
|
||||
|
||||
Banding gives *exact* candidate recall for pHash distances up to 7 (see the comment in
|
||||
`DuplicateGrouper`).
|
||||
|
||||
## Permissions
|
||||
|
||||
| Permission | Why it is needed |
|
||||
|---|---|
|
||||
| `READ_MEDIA_IMAGES` (API 33+) | read the photo library |
|
||||
| `READ_EXTERNAL_STORAGE` (API ≤ 32) | same, on older devices |
|
||||
| `READ_MEDIA_VISUAL_USER_SELECTED` (API 34+) | Android 14 "Selected photos" mode |
|
||||
| `WRITE_EXTERNAL_STORAGE` (**API ≤ 29 only**) | legacy delete path on Android 10 |
|
||||
| `POST_NOTIFICATIONS` (API 33+) | shows the scan-progress notification (optional — the scan runs without it) |
|
||||
| `FOREGROUND_SERVICE` + `FOREGROUND_SERVICE_DATA_SYNC` | keeps a long scan alive while the app is backgrounded |
|
||||
|
||||
Deletion on Android 11+ always goes through the system consent dialog. The app
|
||||
never requests `MANAGE_EXTERNAL_STORAGE`.
|
||||
|
||||
### Scanning in the background
|
||||
|
||||
A scan is owned by `ScanController` (process-scoped, created by the Application), not
|
||||
by the screen's ViewModel. While one is running, `ScanService` holds a `dataSync`
|
||||
foreground service with a progress notification that has a **Cancel** action. So:
|
||||
|
||||
- leaving the app does not kill the scan,
|
||||
- the result survives screen rotation and re-entry,
|
||||
- the user always has a visible way to stop it.
|
||||
|
||||
When `POST_NOTIFICATIONS` is denied the scan still works; the notification is simply
|
||||
not shown.
|
||||
|
||||
## Testing
|
||||
|
||||
`app/src/test` holds JVM unit tests for `core/`: SHA-256 test vectors, dHash bit
|
||||
patterns, Hamming distance, exact/perceptual grouping, aspect-ratio filtering and
|
||||
the reclaimable-bytes maths.
|
||||
|
||||
```bash
|
||||
./gradlew :app:testDebugUnitTest
|
||||
```
|
||||
|
||||
There is no instrumentation test and no emulator in this environment, so UI
|
||||
behaviour is verified on a real device via `adb install` or Play internal testing.
|
||||
|
||||
## Releasing to Google Play
|
||||
|
||||
See [docs/PLAY_RELEASE_CHECKLIST.md](docs/PLAY_RELEASE_CHECKLIST.md) (signing, store
|
||||
listing, Data Safety, release notes, rollout), [docs/STORE_LISTING.md](docs/STORE_LISTING.md)
|
||||
(ready-to-paste listing copy plus the icon / feature graphic generators) and
|
||||
[PRIVACY.md](PRIVACY.md).
|
||||
|
||||
The upload-ready graphics and screenshots live in [`store-assets/`](store-assets/).
|
||||
`tools/make-store-assets.sh` and `tools/make-store-screenshots.sh` regenerate them from the
|
||||
app's own icon vectors and from `adb` captures — always inside the repo, never in `$HOME`.
|
||||
|
||||
User-visible changes are tracked in [CHANGELOG.md](CHANGELOG.md); the Play "What's new"
|
||||
text you type per release is separate and lives only in the Play Console.
|
||||
|
||||
## Localisation
|
||||
|
||||
All user-visible text lives in `app/src/main/res/values/strings.xml`, with proper `<plurals>`
|
||||
where a count can be one, and both the screen and the progress notification resolve their
|
||||
labels through `text/TextRes.kt`. **English is currently the only locale.**
|
||||
|
||||
Adding a language is then a file, not a refactor:
|
||||
|
||||
```
|
||||
app/src/main/res/values-de/strings.xml # same keys, translated values
|
||||
```
|
||||
|
||||
One caveat worth knowing: **Google Play's automatic translation translates your store
|
||||
listing, not the app UI.** Translated UI text has to be supplied in `values-<locale>/`, by a
|
||||
translator or by running the strings file through a translation service and reviewing it.
|
||||
|
||||
Byte units ("KB", "MB") and the duration formatter are intentionally left unlocalised.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Only **images** are scanned — videos are not supported yet.
|
||||
- Near-duplicate detection ignores EXIF rotation, so an image saved physically
|
||||
rotated by 90° will not match its original.
|
||||
- The Aggressive policy may miss some pairs because of the banded candidate search.
|
||||
- The very first scan of a huge library (50k+ photos) takes a while; later scans
|
||||
reuse the hash index and are much faster.
|
||||
|
||||
## Roadmap
|
||||
|
||||
The project is tracked against a milestone plan (M0–M10). What is done, what remains,
|
||||
the current release blockers and the candidate features all live in
|
||||
[docs/ROADMAP.md](docs/ROADMAP.md).
|
||||
|
||||
## Licence
|
||||
|
||||
Proprietary — all rights reserved. Change this if you intend to open-source it.
|
||||
Reference in New Issue
Block a user