fishfacegit/foto-duplicate-finder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add a generated demo photo library for store screenshots

Browse Source 
The results screenshot is a screenshot *of a photo library*, so the committed
capture showed the developer's own photos. tools/make-demo-photos.py draws the
library instead: 22 procedural scenes (sunset, mountains, beach, conifers in
haze, dunes, city skyline, night sky, bokeh) with grain and a vignette, so there
are no people, no third-party content and nothing to license.

The duplicate structure is deliberate, because a folder of unrelated pictures
demonstrates nothing:

- headline group: one scene re-saved at falling quality and size until its
  reclaimable size reaches --target-mb (45 MB). Results are ranked by
  reclaimable bytes, so this group has to out-rank whatever is already on the
  device or a real photo is the first thing back in frame.
- exact pair: two byte-identical files, for the SHA-256 path.
- singles: the other scenes, once each, so the library looks lived-in.

Verified with the app's own engine on the desktop before pushing anything:
22 images -> 2 groups (PERCEPTUAL 14 copies, EXACT 2 copies) with the six
singles left alone. Regeneration is byte-identical, the output path resolves
from __file__ so it runs from any cwd, and nothing is written outside the repo.

demo-photos/ is git-ignored, like fixtures/ and store-assets/raw-screenshots/.
STORE_LISTING.md documents the library, why the headline group has a target
size, the MediaProvider scan_file recipe and the cleanup step; ROADMAP, README
and the release checklist point at it.
This commit is contained in:
Hadrian Burkhardt
2026-10-13 23:34:12 +02:00
parent 1145944659
commit 3581ffc6bb
6 changed files with 534 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/make-demo-photos.py
+450
View File
@@ -0,0 +1,450 @@
#!/usr/bin/env python3
"""Generate a synthetic photo library for store screenshots and demo runs.
Nothing in here is a photograph: every scene is drawn procedurally — gradients, ridges,
silhouettes, grain, vignette — so a screenshot taken over this library can be published
without putting anyone's private pictures in the store listing.
The duplicate structure is deliberate, which is the second reason to generate it: the app
is meant to *find* something, and a folder of unrelated pictures shows nothing.
headline group one scene re-saved at falling quality and size, to a size you set with
--target-mb. Groups are ranked by reclaimable bytes, so this has to
out-rank whatever is already in the real library or a real photo would
be the first thing on the results screen.
exact pair two byte-identical files, for the SHA-256 path
singles one copy each of the other scenes, so the library looks lived-in
Written to demo-photos/foto-dup-demo/ inside the repository (git-ignored, regenerated on
demand):
python3 tools/make-demo-photos.py
adb push demo-photos/foto-dup-demo /sdcard/DCIM/foto-dup-demo
adb shell content call --uri content://media --method scan_file \\
--arg /sdcard/DCIM/foto-dup-demo
`scan_file` is the MediaProvider rescan hook (API 30+); it indexes just that path, so the
rest of the library is untouched. Then open the app, scan, and capture.
"""
from __future__ import annotations
import argparse
import math
import sys
from pathlib import Path
import numpy as np
from PIL import Image, ImageChops, ImageDraw, ImageFilter
REPO = Path(__file__).resolve().parent.parent
OUT = REPO / "demo-photos" / "foto-dup-demo"
# 12 MP, the shape a phone camera writes, so file sizes land in the range real photos do.
FULL = (4032, 3024)
# Scenes are painted at a third of that and upscaled: cheap, and the softness reads as
# "photograph" rather than "illustration".
RENDER = (1344, 1008)
SEED = 20240718
GRAIN = 12
def _rng() -> np.random.Generator:
return np.random.default_rng(SEED)
def _gradient(size: tuple[int, int], stops: list[tuple[float, tuple[int, int, int]]]) -> np.ndarray:
"""Vertical gradient through (position 0..1, (r, g, b)) stops."""
width, height = size
ys = np.linspace(0.0, 1.0, height)
positions = np.array([stop[0] for stop in stops])
channels = [
np.interp(ys, positions, [stop[1][channel] for stop in stops])
for channel in range(3)
]
column = np.stack(channels, axis=-1)
return np.repeat(column[:, None, :], width, axis=1)
def _to_image(array: np.ndarray) -> Image.Image:
return Image.fromarray(np.clip(array, 0, 255).astype(np.uint8), "RGB")
def _ridge(rng: np.random.Generator, width: int, base: float, amplitude: float, roughness: float) -> np.ndarray:
"""A horizon line built from summed sine waves — smooth, never jagged."""
xs = np.linspace(0.0, 1.0, width)
line = np.full(width, base, dtype=np.float64)
for octave in range(4):
line += (amplitude / (2 ** (octave + 1))) * np.sin(
math.tau * roughness * (2**octave) * xs + rng.uniform(0, math.tau)
)
return line
def _fill_ridge(image: Image.Image, line: np.ndarray, colour: tuple[int, int, int]) -> None:
"""Fill everything below a horizon line, i.e. draw a hill or dune."""
width, height = image.size
points = [(x, float(line[x])) for x in range(width)]
points += [(width, height), (0, height)]
ImageDraw.Draw(image).polygon(points, fill=colour)
def _snow(image: Image.Image, line: np.ndarray) -> None:
"""White caps on the high parts of a ridge."""
width, _ = image.size
top, bottom = float(line.min()), float(line.max())
cutoff = top + (bottom - top) * 0.35
draw = ImageDraw.Draw(image, "RGBA")
for x in range(0, width - 6, 4):
if line[x] <= cutoff:
drop = 8 + (cutoff - line[x]) * 0.6
draw.polygon(
((x, line[x]), (x + 6, line[x]), (x + 3, line[x] + drop)),
fill=(250, 252, 255, 235),
)
def _vignette(image: Image.Image, strength: float = 0.24) -> Image.Image:
"""Darken the corners. The cheapest way to stop a gradient reading as vector art."""
width, height = image.size
ys = np.linspace(-1.0, 1.0, height)[:, None]
xs = np.linspace(-1.0, 1.0, width)[None, :]
radius = np.sqrt(xs * xs + ys * ys) / math.sqrt(2.0)
mask = 1.0 - strength * radius**2.2
return _to_image(np.asarray(image, dtype=np.float32) * mask[..., None])
def _grain(image: Image.Image, sigma: int = GRAIN) -> Image.Image:
"""Fine sensor noise. Also, conveniently, what gives a JPEG a realistic file size."""
noise = Image.effect_noise(image.size, sigma).convert("RGB")
return ImageChops.add(image, noise, scale=1.0, offset=-128)
def finish(size: tuple[int, int], painter, rng: np.random.Generator) -> Image.Image:
"""Paint a scene small, blow it up, and make it look photographed."""
scene = painter(RENDER, rng).resize(size, Image.LANCZOS)
scene = scene.filter(ImageFilter.GaussianBlur(0.5))
return _grain(_vignette(scene))
def save(
image: Image.Image,
path: Path,
quality: int,
scale: float = 1.0,
taken: str | None = None,
) -> int:
if scale != 1.0:
size = (int(image.width * scale) // 2 * 2, int(image.height * scale) // 2 * 2)
image = image.resize(size, Image.LANCZOS)
exif = Image.Exif()
exif[0x0112] = 1 # orientation — "as shot"
if taken:
exif[0x9003] = taken
image.save(path, "JPEG", quality=quality, subsampling=2, exif=exif)
return path.stat().st_size
def scene_sunset(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
horizon = int(height * 0.60)
sky = _gradient((width, horizon), [
(0.0, (44, 28, 78)), (0.55, (168, 74, 96)), (0.85, (240, 140, 72)), (1.0, (255, 208, 132)),
])
water = _gradient((width, height - horizon), [
(0.0, (252, 178, 96)), (0.30, (156, 84, 108)), (1.0, (30, 20, 52)),
])
image = _to_image(np.vstack([sky, water]))
draw = ImageDraw.Draw(image, "RGBA")
sun_x = int(width * 0.62)
radius = height * 0.07
draw.ellipse(
(sun_x - radius, horizon - radius * 1.4, sun_x + radius, horizon + radius * 0.6),
fill=(255, 240, 178, 255),
)
# The sun's reflection: a streak that widens as it comes towards the camera.
draw.polygon(
((sun_x - radius * 0.5, horizon), (sun_x + radius * 0.5, horizon),
(sun_x + radius * 2.4, height), (sun_x - radius * 2.4, height)),
fill=(255, 214, 150, 60),
)
image = image.filter(ImageFilter.GaussianBlur(2.5))
_fill_ridge(image, _ridge(rng, width, horizon + height * 0.015, height * 0.045, 1.1), (26, 18, 42))
return image
def scene_mountains(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
horizon = height * 0.72
image = _to_image(_gradient((width, height), [
(0.0, (58, 108, 180)), (0.45, (150, 190, 226)), (0.72, (226, 232, 231)), (1.0, (198, 208, 208)),
]))
for index, (base, amplitude, colour) in enumerate([
(horizon - height * 0.30, height * 0.16, (138, 160, 190)),
(horizon - height * 0.20, height * 0.14, (100, 124, 158)),
(horizon - height * 0.11, height * 0.12, (62, 82, 112)),
]):
line = _ridge(rng, width, base, amplitude, 1.4 + index * 0.8)
_fill_ridge(image, line, colour)
_snow(image, line)
return image.filter(ImageFilter.GaussianBlur(1.1))
def scene_beach(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
sea, sand = int(height * 0.55), int(height * 0.74)
image = _to_image(np.vstack([
_gradient((width, sea), [(0.0, (86, 158, 214)), (0.7, (170, 212, 236)), (1.0, (238, 246, 240))]),
_gradient((width, sand - sea), [(0.0, (34, 110, 140)), (0.6, (58, 156, 176)), (1.0, (150, 208, 208))]),
_gradient((width, height - sand), [(0.0, (238, 220, 186)), (1.0, (206, 178, 138))]),
]))
draw = ImageDraw.Draw(image, "RGBA")
for index in range(5): # foam where the water meets the sand
y = sand - 4 + index * 5
inset = width * rng.uniform(0.0, 0.25)
draw.line((inset, y, width - inset, y + rng.uniform(-3, 3)), fill=(255, 255, 255, 90), width=3)
image = image.filter(ImageFilter.GaussianBlur(1.6))
# The blur above rebound `image`; the umbrella has to be drawn through a fresh handle.
draw = ImageDraw.Draw(image, "RGBA")
pole_x = int(width * 0.30)
canopy_y = sand - (height - sand) * 0.20
apex_y = sand - (height - sand) * 0.44
half = width * 0.11
draw.line((pole_x, sand + (height - sand) * 0.85, pole_x, sand - (height - sand) * 0.25),
fill=(228, 226, 220, 255), width=6)
draw.polygon(
((pole_x - half, canopy_y), (pole_x + half, canopy_y), (pole_x, apex_y)),
fill=(226, 92, 78, 255),
)
# One pale panel, so it reads as a stripy beach umbrella rather than a red triangle.
draw.polygon(
((pole_x - half * 0.35, canopy_y), (pole_x + half * 0.05, canopy_y), (pole_x, apex_y)),
fill=(244, 240, 232, 255),
)
draw.ellipse(
(pole_x - width * 0.07, sand + (height - sand) * 0.80,
pole_x + width * 0.07, sand + (height - sand) * 0.86),
fill=(60, 48, 40, 70),
)
return image
def scene_forest(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
"""Foggy conifer hillside: three bands of trees, each behind a sheet of haze."""
width, height = size
image = _to_image(_gradient((width, height), [
(0.0, (198, 216, 208)), (0.45, (152, 180, 170)), (1.0, (86, 112, 100)),
]))
mist = Image.new("RGB", (width, height), (224, 234, 230))
for index, (base, tree, colour) in enumerate([
(height * 0.46, height * 0.09, (120, 146, 136)),
(height * 0.62, height * 0.13, (88, 114, 102)),
(height * 0.80, height * 0.19, (48, 70, 60)),
]):
if index:
# The haze belongs *in front of* the trees drawn before it.
image = Image.blend(image, mist, 0.28)
draw = ImageDraw.Draw(image)
for _ in range(30):
x = rng.uniform(-width * 0.02, width * 1.02)
top = base - rng.uniform(tree * 0.55, tree * 1.45)
half = rng.uniform(width * 0.010, width * 0.026)
draw.polygon(((x - half, base), (x + half, base), (x, top)), fill=colour)
image = image.filter(ImageFilter.GaussianBlur(1.0))
_fill_ridge(image, _ridge(rng, width, height * 0.94, height * 0.015, 0.9), (34, 50, 42))
return image
def scene_dunes(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
image = _to_image(_gradient((width, height), [
(0.0, (96, 146, 196)), (0.35, (214, 190, 158)), (0.55, (238, 206, 158)), (1.0, (198, 150, 96)),
]))
for index, colour in enumerate([(240, 214, 170), (226, 190, 138), (204, 162, 108), (176, 132, 84)]):
_fill_ridge(
image,
_ridge(rng, width, height * (0.45 + index * 0.13), height * 0.03, 0.6 + index * 0.4),
colour,
)
return image.filter(ImageFilter.GaussianBlur(1.0))
def scene_city(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
horizon = int(height * 0.72)
image = _to_image(_gradient((width, horizon), [
(0.0, (44, 44, 96)), (0.55, (150, 96, 128)), (0.85, (232, 138, 96)), (1.0, (252, 196, 132)),
]))
image.paste(
_to_image(_gradient((width, height - horizon), [(0.0, (44, 38, 62)), (1.0, (16, 14, 28))])),
(0, horizon),
)
draw = ImageDraw.Draw(image, "RGBA")
draw.ellipse(
(width * 0.70, height * 0.12, width * 0.78, height * 0.12 + width * 0.08),
fill=(250, 244, 220, 255),
)
x = 0.0
while x < width:
building = rng.uniform(width * 0.03, width * 0.075)
top = rng.uniform(height * 0.28, height * 0.60)
draw.rectangle((x, top, x + building, horizon), fill=(34, 32, 52, 255))
for wy in np.arange(top + 8, horizon - 6, 14): # lit windows
for wx in np.arange(x + 6, x + building - 4, 12):
if rng.random() < 0.45:
draw.rectangle((wx, wy, wx + 4, wy + 6), fill=(255, 226, 150, 255))
x += building + rng.uniform(2, 10)
return image.filter(ImageFilter.GaussianBlur(0.9))
def scene_night(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
image = _to_image(_gradient((width, height), [
(0.0, (6, 10, 30)), (0.5, (14, 22, 54)), (0.78, (30, 40, 74)), (1.0, (12, 16, 34)),
]))
# The milky way: a shallow diagonal band of blurred noise, screened over the sky. Kept dim
# and wide — a narrow bright band reads as a light leak, not a galaxy.
band = np.linspace(0.0, 1.0, width)[None, :] * height * 0.22 + height * 0.12
distance = np.abs(np.arange(height)[:, None] - band)
cloud = np.clip(1.0 - distance / (height * 0.20), 0, 1) * rng.random((height, width))
nebula = Image.fromarray((cloud * 120).astype(np.uint8), "L").filter(ImageFilter.GaussianBlur(11))
image = ImageChops.screen(
image,
Image.merge("RGB", (
nebula,
nebula.point(lambda v: int(v * 0.85)),
nebula.point(lambda v: min(255, int(v * 1.3))),
)),
)
draw = ImageDraw.Draw(image)
for _ in range(320):
x, y = rng.uniform(0, width), rng.uniform(0, height * 0.80)
dot = int(rng.choice([1, 1, 1, 2, 3]))
brightness = int(rng.uniform(140, 255))
draw.rectangle((x, y, x + dot, y + dot), fill=(brightness, brightness, min(255, brightness + 20)))
image = image.filter(ImageFilter.GaussianBlur(0.6))
_fill_ridge(image, _ridge(rng, width, height * 0.84, height * 0.05, 1.4), (6, 8, 18))
return image
def scene_bokeh(size: tuple[int, int], rng: np.random.Generator) -> Image.Image:
width, height = size
palette = [(246, 176, 92), (232, 106, 132), (120, 176, 232), (238, 226, 178), (168, 128, 220)]
image = Image.new("RGB", (width, height), (26, 16, 34))
draw = ImageDraw.Draw(image, "RGBA")
for _ in range(90): # out-of-focus highlights
x, y = rng.uniform(0, width), rng.uniform(0, height)
radius = rng.uniform(width * 0.015, width * 0.09)
colour = palette[int(rng.integers(len(palette)))]
draw.ellipse((x - radius, y - radius, x + radius, y + radius),
fill=colour + (int(rng.uniform(40, 110)),))
image = image.filter(ImageFilter.GaussianBlur(9))
draw = ImageDraw.Draw(image, "RGBA")
for _ in range(24): # and a few that landed in focus
x, y = rng.uniform(0, width), rng.uniform(0, height)
radius = rng.uniform(width * 0.006, width * 0.03)
colour = palette[int(rng.integers(len(palette)))]
draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=colour + (190,))
return image.filter(ImageFilter.GaussianBlur(3))
SCENES = {
"sunset": scene_sunset,
"mountains": scene_mountains,
"beach": scene_beach,
"forest": scene_forest,
"dunes": scene_dunes,
"city": scene_city,
"night": scene_night,
"bokeh": scene_bokeh,
}
# Camera-roll names: what a phone writes, plus a suffix for the copies it keeps.
HEADLINE = "IMG_20240718_204512"
HEADLINE_TAKEN = "2024:07:18 20:45:12"
# Re-saves of one photo, in the order a real camera roll collects them: exports at falling
# quality, then the downscaled shares. The loop stops as soon as --target-mb is reached, so a
# bigger real library just means more copies.
HEADLINE_VARIANTS = [
(95, 1.0), (92, 1.0), (90, 1.0), (88, 1.0), (85, 1.0), (82, 1.0), (80, 1.0),
(78, 1.0), (75, 1.0), (90, 0.80), (86, 0.66), (82, 0.50),
# ...and the exports a camera roll keeps collecting afterwards.
(94, 1.0), (89, 1.0), (84, 1.0), (81, 1.0), (79, 1.0), (77, 1.0),
(88, 0.90), (84, 0.72), (80, 0.60), (78, 0.44), (76, 0.33),
]
PAIR = ("IMG_20240315_140122", "forest", 88, 0.5, "2024:03:15 14:01:22")
SINGLES = [
("IMG_20240405_091243", "mountains", "2024:04:05 09:12:43"),
("IMG_20240519_134500", "beach", "2024:05:19 13:45:00"),
("IMG_20240628_182004", "dunes", "2024:06:28 18:20:04"),
("IMG_20240810_213055", "city", "2024:08:10 21:30:55"),
("IMG_20240824_231512", "night", "2024:08:24 23:15:12"),
("IMG_20240909_193044", "bokeh", "2024:09:09 19:30:44"),
]
def reclaimable(sizes: list[int]) -> float:
"""What the app would report: everything except the largest copy, in MB."""
return (sum(sizes) - max(sizes)) / 1_000_000
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--out", type=Path, default=OUT,
help=f"output directory (default: {OUT})")
parser.add_argument("--target-mb", type=float, default=45.0,
help="reclaimable size the headline group must reach (default: 45)")
args = parser.parse_args()
rng = _rng()
args.out.mkdir(parents=True, exist_ok=True)
for stale in sorted(args.out.glob("*.jpg")):
stale.unlink()
# 1. The headline group. Results are ranked by reclaimable bytes, so this one has to
# out-rank everything in the real library or a real photo is the first thing on screen.
headline = finish(FULL, SCENES["sunset"], rng)
sizes: list[int] = []
for index, (quality, scale) in enumerate(HEADLINE_VARIANTS):
name = f"{HEADLINE}.jpg" if index == 0 else f"{HEADLINE}-{index}.jpg"
sizes.append(save(headline, args.out / name, quality, scale, HEADLINE_TAKEN))
if reclaimable(sizes) >= args.target_mb:
break
print(f"headline group: {len(sizes)} copies, {reclaimable(sizes):.1f} MB reclaimable "
f"(target {args.target_mb:g} MB)")
if reclaimable(sizes) < args.target_mb:
print(f"warning: out of variants at {reclaimable(sizes):.1f} MB — the group may not "
f"out-rank the real library. Add entries to HEADLINE_VARIANTS or lower "
f"--target-mb.", file=sys.stderr)
# 2. Two byte-identical files, for the SHA-256 path.
name, scene, quality, scale, taken = PAIR
original = args.out / f"{name}.jpg"
save(finish(FULL, SCENES[scene], rng), original, quality, scale, taken)
(args.out / f"{name}-copy.jpg").write_bytes(original.read_bytes())
# 3. One of each remaining scene, so the library looks lived-in.
for name, scene, taken in SINGLES:
save(finish(FULL, SCENES[scene], rng), args.out / f"{name}.jpg", 92, 1.0, taken)
written = sorted(args.out.glob("*.jpg"))
total = sum(path.stat().st_size for path in written)
print(f"wrote {len(written)} demo photos ({total / 1e6:.1f} MB) to {args.out}")
for path in written:
print(f" {path.stat().st_size:>9,} {path.name}")
shown = args.out.relative_to(REPO) if args.out.is_relative_to(REPO) else args.out
print("\nPush them to the device and index just that folder (API 30+):")
print(f" adb push {shown} /sdcard/DCIM/foto-dup-demo")
print(" adb shell content call --uri content://media --method scan_file \\")
print(" --arg /sdcard/DCIM/foto-dup-demo")
if __name__ == "__main__":
main()