Initial commit: Foto Dup Finder 1.0.1
An Android duplicate-photo finder. Pure-Kotlin core (SHA-256 + dHash hashing, exact and perceptual union-find grouping, three sensitivity presets) with MediaStore access, a persistent SQLite hash index and a tiered ScanEngine behind a Jetpack Compose UI. No INTERNET permission, no ads, no analytics, no account. Ships versionCode 2 / versionName 1.0.1, carrying the scan-stall hardening (BoundedCall breaker) and verified against a 5,363-photo device library. 80 JVM unit tests, lint clean. Release paperwork is in docs/ (Play checklist, store listing, edge-case testing), the upload-ready Play graphics and screenshots in store-assets/, and the generators for those assets in tools/.
This commit is contained in:
+165
@@ -0,0 +1,165 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to Foto Dup Finder are documented in this file.
|
||||
|
||||
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the
|
||||
project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
`versionCode` and `versionName` are defined in `app/build.gradle.kts`. Each Play
|
||||
upload needs a **higher `versionCode`** than every previous upload.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.0.1] — 2026-10-13 · versionCode 2
|
||||
|
||||
### Fixed
|
||||
|
||||
- **A stalled media provider could still make a scan look frozen.** The per-call deadline above
|
||||
stops a *single* stuck file, but not a provider that has stopped answering altogether — on the
|
||||
test device `MediaProvider` was restarting repeatedly during the scan. Every remaining photo then
|
||||
waits out the full deadline, so the scan grinds at roughly one photo per timeout and looks hung,
|
||||
while each abandoned call parks another thread. `data/BoundedCall` now counts abandoned calls:
|
||||
after three in a row with nothing succeeding in between it treats the provider as **stalled** and
|
||||
lets the remaining calls fail fast, so the phase reaches its end and the unreadable files are
|
||||
reported instead of the scan appearing to freeze. The cooldown scales with the call deadline, and
|
||||
a successful call clears it, so a brief `MediaProvider` restart is picked up again. Pinned by
|
||||
`BoundedCallTest` (50 blocking calls park a bounded number of threads; the breaker clears once
|
||||
the provider answers) and by an end-to-end test that a provider read which never returns cannot
|
||||
freeze the scan.
|
||||
|
||||
- **One unresponsive thumbnail could freeze the whole scan.** The perceptual pass asks the media
|
||||
provider for a thumbnail (`ContentResolver.loadThumbnail`). On the test device that call never
|
||||
returned for a single, perfectly valid HEIC — the provider hung inside its own decode — and the
|
||||
scan sat at 99 % of "Looking for visually similar photos…" for ever, at 0 % CPU. The per-photo
|
||||
`withTimeoutOrNull` could not help: coroutine cancellation is **cooperative**, so it cannot
|
||||
interrupt a thread parked in a binder transaction. Every provider call (signature read, content
|
||||
hash, thumbnail/fallback decode) now runs under a hard deadline (`data/BoundedCall`), and a call
|
||||
that overruns is abandoned — that one photo is skipped and counted among the unreadable ones
|
||||
instead of stopping the scan. `BoundedCallTest` pins the behaviour.
|
||||
|
||||
## [1.0.0] — 2026-10-13 · versionCode 1
|
||||
|
||||
First public release.
|
||||
|
||||
### Added
|
||||
|
||||
- Exact duplicate detection: files are bucketed by size, refined with a cheap
|
||||
`SHA-256(size + first & last 32 KB)` signature, then confirmed with a full
|
||||
SHA-256 content hash.
|
||||
- Near-duplicate detection via a 64-bit perceptual difference hash (`dHash`) on a
|
||||
9×8 grayscale downscale, clustered by Hamming distance with union-find.
|
||||
- Three sensitivity presets: Exact copies only / Balanced (default) / Aggressive.
|
||||
- Tiered, CPU-parallel scan pipeline with a persistent SQLite hash index, so repeat
|
||||
scans only re-read files that changed.
|
||||
- Results UI grouped by duplicate set, with thumbnails, a "BEST" marker and
|
||||
per-group All / None selection.
|
||||
- "Select all but best" helper that keeps the largest (then newest) copy.
|
||||
- Safe deletion through the system consent dialog
|
||||
(`MediaStore.createDeleteRequest`) or the system trash
|
||||
(`createTrashRequest`); legacy direct delete on Android 10.
|
||||
- Scans run in a `dataSync` **foreground service**, so a long scan keeps going while
|
||||
the app is backgrounded, with a progress notification that can cancel it.
|
||||
- Android 13/14 permission handling, including the partial "Selected photos" mode.
|
||||
- Material 3 UI with dynamic colour and dark-mode support.
|
||||
- No ads, no accounts, no analytics and no network permission.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Localisation groundwork (M8).** Every user-visible string moved out of Kotlin into
|
||||
`res/values/strings.xml`, with real `<plurals>` where the count can be one ("1 duplicate"
|
||||
vs "3 duplicates", "1 bit" vs "5 bits"). The screen and the progress notification share a
|
||||
single enum-to-resource mapping (`text.labelRes()`) so their wording cannot drift apart.
|
||||
English is currently the only locale — but note that **Play's automatic translation covers
|
||||
the store listing, not the app UI**, so this is the prerequisite for translating the UI.
|
||||
|
||||
- Near-duplicate detection now uses a **DCT perceptual hash (pHash)** plus **per-channel
|
||||
RGB difference hashes**. A pair must satisfy both. Measured on the benchmark harness,
|
||||
pHash alone recovered 0.683 of duplicates at zero false positives versus 0.530 for the
|
||||
previous dHash, and joint accuracy was 0.867 F1 versus 0.738.
|
||||
- The pHash distance cut is **fitted to each library** (`AdaptiveThreshold`) instead of
|
||||
being a fixed constant, because unrelated pairs grow with n² and a constant cannot hold
|
||||
across library sizes.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Scan hang at the head/tail read (real root cause).** The byte-reading loops treated only a
|
||||
negative result as end-of-stream. A non-blocking, pipe-backed file descriptor — which some media
|
||||
providers hand back — reports **0** ("no bytes yet") instead, so `while (buffer.hasRemaining())`
|
||||
spun forever and the scan froze mid-phase, pegging a CPU core. `PhotoHasher.readAt`,
|
||||
`readAtMost` and the streaming SHA-256 now all treat a non-positive read as the end of the data.
|
||||
Two watchdog tests (daemon thread + join timeout) make a regression fail rather than hang.
|
||||
- Note on the per-photo timeout added just before: coroutine cancellation is cooperative, so it
|
||||
bounds a *slow* file but cannot interrupt a call blocked in a native read. The loop guard above
|
||||
is the actual fix for a stuck descriptor; the timeout stays as a bound on slow work.
|
||||
|
||||
- **A slow scan tail looked like a freeze.** Progress was only reported every `1/n` of a phase and
|
||||
then at completion, so the final ~1 % of the library (up to ~50 photos) produced no update at
|
||||
all — a stall of a minute looked identical to a hang. Progress now keeps updating through the
|
||||
last interval.
|
||||
- **One pathological file could stall a whole scan.** Each per-photo step now runs under a 15 s
|
||||
budget, so a corrupt entry a provider blocks on, or an unusually expensive HEIC/RAW decode, is
|
||||
abandoned and reported as unreadable instead of freezing the scan.
|
||||
|
||||
- **Hardened file handling (M7).** Every per-file step now sits behind a single failure
|
||||
boundary, so a corrupt, truncated, zero-byte, RAW or outright hostile file is skipped
|
||||
instead of aborting the scan or crashing the app. Coroutine cancellation still propagates
|
||||
through it, a failed index write no longer discards a scan that already completed, and the
|
||||
number of unreadable files is reported in the results rather than passing silently.
|
||||
- Decode arithmetic moved to `core/DecodeMath` with division rather than multiplication, so a
|
||||
crafted image declaring 100 000 x 100 000 cannot overflow the sample-size calculation.
|
||||
|
||||
- The per-group button now counts the copies **actually ticked in that row** rather than the
|
||||
app's suggestion, and acts on exactly what it counts. With BEST + DUP1 + DUP2, unticking
|
||||
DUP2 now reads "Remove 1 duplicate" instead of still claiming 2. When nothing in a row is
|
||||
ticked the button says so and is disabled.
|
||||
|
||||
- **EXIF orientation** is now applied on the fallback decode path. `BitmapFactory` ignores
|
||||
EXIF while the media provider's thumbnail does not, so the same photo could hash
|
||||
differently depending on which path served it — and a thumbnail-cache eviction between
|
||||
scans could leave a cached hash that no longer matched.
|
||||
- Candidate-pair enumeration no longer keeps a global set of compared pairs, which for a
|
||||
20 000-photo library meant millions of boxed `Long`s. Memory is now bounded by the
|
||||
largest neighbour set.
|
||||
|
||||
- The bottom action bar no longer overlaps the system navigation bar (it now applies
|
||||
`navigationBarsPadding()`); previously the trash/delete button sat underneath the
|
||||
gesture handle or the 3-button bar on edge-to-edge devices.
|
||||
- Deleting one group no longer clears a selection made in a different group.
|
||||
- The *BEST* badge, "select all but best" and the per-group delete all use one rule
|
||||
(`DuplicateGroup.suggestedKeep()`), so they can no longer disagree.
|
||||
- Only one system confirmation dialog can be in flight; rapidly tapping a delete button
|
||||
no longer queues up several approvals to dismiss one after another.
|
||||
|
||||
### Added (UI)
|
||||
|
||||
- **Two-step removal.** "Remove N duplicates" on a group row (or "Remove all selected" in
|
||||
the action bar) only *stages* photos: the row disappears from the list immediately and
|
||||
**nothing is deleted**. A separate **Apply** button performs the real removal, so on
|
||||
Android 11+ there is **one** system confirmation for the whole batch instead of one
|
||||
dialog per group. **Discard** undoes all staging.
|
||||
- The list always shows what is *still* duplicated: staged copies are taken out of their
|
||||
group, and a group left with fewer than two copies is dropped entirely.
|
||||
- The copy marked *BEST* is always shown **first** in its group
|
||||
(`DuplicateGroup.displayOrder()`), so it is visible as a reference even when the
|
||||
group is collapsed. Previously it could be scrolled out of view.
|
||||
|
||||
### Performance
|
||||
|
||||
- Exact-duplicate hashing now runs **only** for photos that share a file size with
|
||||
at least one other photo. In a typical library most photos have a unique size, so
|
||||
for those no file is opened at all.
|
||||
- The perceptual hash is taken from the media provider's **thumbnail cache**
|
||||
(`loadThumbnail`) instead of decoding the full-resolution original, with a heavily
|
||||
subsampled decode as a fallback.
|
||||
- The head and tail bytes of a file are read through a **single** file descriptor.
|
||||
- Hashing runs in small batches, reducing coroutine overhead from one coroutine per
|
||||
photo to one per batch.
|
||||
- The hash index is written with a compiled SQLite statement.
|
||||
- Per-stage timings are shown in the results summary so slow stages are visible.
|
||||
|
||||
### Known limitations
|
||||
|
||||
- Images only — video files are not scanned.
|
||||
- `dHash` ignores EXIF rotation, so a physically rotated copy will not match.
|
||||
- The Aggressive preset may miss a few pairs because of the banded candidate search.
|
||||
- The first scan of a very large library takes a while; later scans reuse the index.
|
||||
Reference in New Issue
Block a user