Keep the search index current without re-walking the volume
A search everywhere rebuilt the whole index once it was older than fifteen minutes, so every session paid for a walk of the entire volume - tens of seconds of reading before a single result - even though almost nothing had changed. StorageSource gains names(), which lists a folder's names without reading a size and a date for every child. That is the difference between a walk and a catch-up: list() costs three stat round trips per entry on a phone's mounted storage, and a name index does not need any of them for entries it already knows. SearchIndexer.refresh() compares the names a folder holds now against the names already indexed and reads metadata only for names it has never seen, writing nothing at all for a folder that has not changed. The search itself now answers first and catches up behind the answer: whatever the index knows is shown immediately, and the pass that makes it current runs afterwards and answers again, which is the whole of "I just downloaded it, so why is it not here?". A full walk is now the exception - a volume never indexed, one whose rows were built under the other hidden-files preference, one not walked in full for a week, or the explicit Rebuild - because a catch-up compares names and so cannot see a file swapped for a different file of the same name, nor what a preference change means for rows that were never counted. Two timestamps per volume keep that distinction, which is a schema change: v2 adds full_at and hidden_included with a real migration that keeps the rows, since throwing them away would cost the next search the walk it just avoided. A folder that cannot be read now answers null rather than empty, and a catch-up leaves what it knows about it alone: concluding "everything inside has gone" from a transient failure is how an index loses things that are still there. The tests assert what a pass *reads*, which is the point of the change: a current index makes no call to the storage at all, a stale one reads only the names it has never seen, and a walk followed by a catch-up over a real temporary tree keeps the index level with files and folders added and deleted between them. AllFilesSourceTest covers the new names() against the real filesystem, including the null for an unreadable folder. 288 JVM tests, lint clean. No device pass this time: /dev/kvm disappeared from this environment partway through, so the emulator could not be started again. The device checks for the index itself were made in the previous round.
This commit is contained in:
@@ -168,6 +168,33 @@ upload needs a **higher `versionCode`** than every previous upload.
|
||||
path in the row, and a rebuild reported progress and re-answered the query when it finished.
|
||||
Two honesty bugs found by watching it: the empty state claimed "Nothing matches" *while* the
|
||||
index was still being written, and two progress bars stacked on each other.
|
||||
- **…and it no longer re-walks the volume to stay current.** A search everywhere used to
|
||||
rebuild the whole index once it was older than fifteen minutes, which meant paying for a walk
|
||||
of the entire volume every session — on a phone that is tens of seconds of reading before a
|
||||
single result. Three changes remove almost all of it. **The answer comes first**: whatever
|
||||
the index already knows is shown immediately, and it is brought up to date *behind* that
|
||||
answer, so no walk is ever between the user and their results. **Staying current is now a
|
||||
catch-up, not a walk**: `StorageSource.names` lists a folder's names without reading a size
|
||||
and a date for every child — three `stat` round trips per entry is what made the walk slow —
|
||||
and the catch-up compares those names against the ones already indexed, reading metadata only
|
||||
for names it has never seen. **A walk is now the exception**: it runs for a volume that has
|
||||
never been indexed, for one whose rows were built under the other hidden-files preference
|
||||
(the rows mean different things, so they cannot be caught up), for one not walked in full for
|
||||
a week, or when the user taps Rebuild. A catch-up compares names alone, so those are exactly
|
||||
the things it structurally cannot see: a file swapped for a different file of the same name,
|
||||
and the preference's effect on what was ever counted. Two timestamps per volume keep that
|
||||
honest, which is a schema change — v2 adds `full_at` and `hidden_included` with a real
|
||||
migration that keeps the rows, because throwing them away would cost the next search a walk
|
||||
it need not do. A folder that cannot be read now answers `null` rather than "empty", and a
|
||||
catch-up leaves what it knows about it exactly as it was: deleting rows on a transient
|
||||
failure is how an index loses things that are still there. Covered by tests that assert what
|
||||
a pass *reads* rather than only what it writes — a fresh index makes no call to the storage
|
||||
at all, a stale one reads only the new names, and a walk and a catch-up over a real temporary
|
||||
tree keep the index level with files added and deleted between them. **288 JVM tests, lint
|
||||
clean.** No device pass this time: the emulator's `/dev/kvm` disappeared from this
|
||||
environment partway through (the previous round's device checks had already run), so this
|
||||
round's evidence is the suite plus `AllFilesSourceTest` exercising the real
|
||||
`File.list()` path — which is the one piece a fake cannot stand in for.
|
||||
|
||||
- **Six more kinds of file can be read in the app instead of guessed at.** Images and text had
|
||||
a preview and nothing else did, so a zip, a PDF, a video, a song or an APK offered only
|
||||
|
||||
Reference in New Issue
Block a user