foto-duplicate-finder/CHANGELOG.md

Changelog

All notable changes to Foto Dup Finder are documented in this file.

The format follows Keep a Changelog, and the project uses Semantic Versioning.

versionCode and versionName are defined in app/build.gradle.kts. Each Play upload needs a higher versionCode than every previous upload.

[Unreleased]

[1.0.1] — 2026-10-13 · versionCode 2

Fixed

• A stalled media provider could still make a scan look frozen. The per-call deadline above stops a single stuck file, but not a provider that has stopped answering altogether — on the test device MediaProvider was restarting repeatedly during the scan. Every remaining photo then waits out the full deadline, so the scan grinds at roughly one photo per timeout and looks hung, while each abandoned call parks another thread. data/BoundedCall now counts abandoned calls: after three in a row with nothing succeeding in between it treats the provider as stalled and lets the remaining calls fail fast, so the phase reaches its end and the unreadable files are reported instead of the scan appearing to freeze. The cooldown scales with the call deadline, and a successful call clears it, so a brief MediaProvider restart is picked up again. Pinned by BoundedCallTest (50 blocking calls park a bounded number of threads; the breaker clears once the provider answers) and by an end-to-end test that a provider read which never returns cannot freeze the scan.

• One unresponsive thumbnail could freeze the whole scan. The perceptual pass asks the media provider for a thumbnail (ContentResolver.loadThumbnail). On the test device that call never returned for a single, perfectly valid HEIC — the provider hung inside its own decode — and the scan sat at 99 % of "Looking for visually similar photos…" for ever, at 0 % CPU. The per-photo withTimeoutOrNull could not help: coroutine cancellation is cooperative, so it cannot interrupt a thread parked in a binder transaction. Every provider call (signature read, content hash, thumbnail/fallback decode) now runs under a hard deadline (data/BoundedCall), and a call that overruns is abandoned — that one photo is skipped and counted among the unreadable ones instead of stopping the scan. BoundedCallTest pins the behaviour.

[1.0.0] — 2026-10-13 · versionCode 1

First public release.

Added

• Exact duplicate detection: files are bucketed by size, refined with a cheap SHA-256(size + first & last 32 KB) signature, then confirmed with a full SHA-256 content hash.
• Near-duplicate detection via a 64-bit perceptual difference hash (dHash) on a 9×8 grayscale downscale, clustered by Hamming distance with union-find.
• Three sensitivity presets: Exact copies only / Balanced (default) / Aggressive.
• Tiered, CPU-parallel scan pipeline with a persistent SQLite hash index, so repeat scans only re-read files that changed.
• Results UI grouped by duplicate set, with thumbnails, a "BEST" marker and per-group All / None selection.
• "Select all but best" helper that keeps the largest (then newest) copy.
• Safe deletion through the system consent dialog (MediaStore.createDeleteRequest) or the system trash (createTrashRequest); legacy direct delete on Android 10.
• Scans run in a dataSync foreground service, so a long scan keeps going while the app is backgrounded, with a progress notification that can cancel it.
• Android 13/14 permission handling, including the partial "Selected photos" mode.
• Material 3 UI with dynamic colour and dark-mode support.
• No ads, no accounts, no analytics and no network permission.

Changed

• Localisation groundwork (M8). Every user-visible string moved out of Kotlin into res/values/strings.xml, with real <plurals> where the count can be one ("1 duplicate" vs "3 duplicates", "1 bit" vs "5 bits"). The screen and the progress notification share a single enum-to-resource mapping (text.labelRes()) so their wording cannot drift apart. English is currently the only locale — but note that Play's automatic translation covers the store listing, not the app UI, so this is the prerequisite for translating the UI.

• Near-duplicate detection now uses a DCT perceptual hash (pHash) plus per-channel RGB difference hashes. A pair must satisfy both. Measured on the benchmark harness, pHash alone recovered 0.683 of duplicates at zero false positives versus 0.530 for the previous dHash, and joint accuracy was 0.867 F1 versus 0.738.

• The pHash distance cut is fitted to each library (AdaptiveThreshold) instead of being a fixed constant, because unrelated pairs grow with n² and a constant cannot hold across library sizes.

Fixed

• Scan hang at the head/tail read (real root cause). The byte-reading loops treated only a negative result as end-of-stream. A non-blocking, pipe-backed file descriptor — which some media providers hand back — reports 0 ("no bytes yet") instead, so while (buffer.hasRemaining()) spun forever and the scan froze mid-phase, pegging a CPU core. PhotoHasher.readAt, readAtMost and the streaming SHA-256 now all treat a non-positive read as the end of the data. Two watchdog tests (daemon thread + join timeout) make a regression fail rather than hang.

• Note on the per-photo timeout added just before: coroutine cancellation is cooperative, so it bounds a slow file but cannot interrupt a call blocked in a native read. The loop guard above is the actual fix for a stuck descriptor; the timeout stays as a bound on slow work.

• A slow scan tail looked like a freeze. Progress was only reported every 1/n of a phase and then at completion, so the final ~1 % of the library (up to ~50 photos) produced no update at all — a stall of a minute looked identical to a hang. Progress now keeps updating through the last interval.

• One pathological file could stall a whole scan. Each per-photo step now runs under a 15 s budget, so a corrupt entry a provider blocks on, or an unusually expensive HEIC/RAW decode, is abandoned and reported as unreadable instead of freezing the scan.

• Hardened file handling (M7). Every per-file step now sits behind a single failure boundary, so a corrupt, truncated, zero-byte, RAW or outright hostile file is skipped instead of aborting the scan or crashing the app. Coroutine cancellation still propagates through it, a failed index write no longer discards a scan that already completed, and the number of unreadable files is reported in the results rather than passing silently.

• Decode arithmetic moved to core/DecodeMath with division rather than multiplication, so a crafted image declaring 100 000 x 100 000 cannot overflow the sample-size calculation.

• The per-group button now counts the copies actually ticked in that row rather than the app's suggestion, and acts on exactly what it counts. With BEST + DUP1 + DUP2, unticking DUP2 now reads "Remove 1 duplicate" instead of still claiming 2. When nothing in a row is ticked the button says so and is disabled.

• EXIF orientation is now applied on the fallback decode path. BitmapFactory ignores EXIF while the media provider's thumbnail does not, so the same photo could hash differently depending on which path served it — and a thumbnail-cache eviction between scans could leave a cached hash that no longer matched.

• Candidate-pair enumeration no longer keeps a global set of compared pairs, which for a 20 000-photo library meant millions of boxed Longs. Memory is now bounded by the largest neighbour set.

• The bottom action bar no longer overlaps the system navigation bar (it now applies navigationBarsPadding()); previously the trash/delete button sat underneath the gesture handle or the 3-button bar on edge-to-edge devices.

• Deleting one group no longer clears a selection made in a different group.

• The BEST badge, "select all but best" and the per-group delete all use one rule (DuplicateGroup.suggestedKeep()), so they can no longer disagree.

• Only one system confirmation dialog can be in flight; rapidly tapping a delete button no longer queues up several approvals to dismiss one after another.

Added (UI)

• Two-step removal. "Remove N duplicates" on a group row (or "Remove all selected" in the action bar) only stages photos: the row disappears from the list immediately and nothing is deleted. A separate Apply button performs the real removal, so on Android 11+ there is one system confirmation for the whole batch instead of one dialog per group. Discard undoes all staging.
• The list always shows what is still duplicated: staged copies are taken out of their group, and a group left with fewer than two copies is dropped entirely.
• The copy marked BEST is always shown first in its group (DuplicateGroup.displayOrder()), so it is visible as a reference even when the group is collapsed. Previously it could be scrolled out of view.

Performance

• Exact-duplicate hashing now runs only for photos that share a file size with at least one other photo. In a typical library most photos have a unique size, so for those no file is opened at all.
• The perceptual hash is taken from the media provider's thumbnail cache (loadThumbnail) instead of decoding the full-resolution original, with a heavily subsampled decode as a fallback.
• The head and tail bytes of a file are read through a single file descriptor.
• Hashing runs in small batches, reducing coroutine overhead from one coroutine per photo to one per batch.
• The hash index is written with a compiled SQLite statement.
• Per-stage timings are shown in the results summary so slow stages are visible.

Known limitations

• Images only — video files are not scanned.
• dHash ignores EXIF rotation, so a physically rotated copy will not match.
• The Aggressive preset may miss a few pairs because of the banded candidate search.
• The first scan of a very large library takes a while; later scans reuse the index.