Draw a large folder while it is being read, instead of after
Two seconds of readdir and stat cannot be made much cheaper - each is a round trip to another process - so the folder is now read in batches and drawn as it arrives. StorageSource.listBatches hands over 200 entries at a time; AllFilesSource implements it with Files.newDirectoryStream, which is lazy where File.listFiles materialised the whole directory first; the default implementation is one batch, so the SAF backend and every test fake are unchanged. BrowserViewModel.load merges each batch into the list already on screen with core/SortedMerge, which is O(n+k) rather than re-sorting the folder on every batch, and keeps the rows in the user's order at every moment instead of shuffling them into place when the last batch lands. loading stays true until the last batch, so the progress bar the screen already draws sits above rows that are already useful. publishEntries is separate from recompute for that reason: the merge has done the ordering, and the batch path only has to apply the hidden filter. Tests: four for the merge (including ties not jumping ahead) and one for the batched read, which asserts 250 files arrive as 100+100+50 covering every entry exactly once - the thing a streaming read is easiest to get wrong about. PERFORMANCE.md records the expected effect and states plainly that it is not measured yet, since confirming it needs the 20000-file fixture rebuilt.
This commit is contained in:
@@ -160,6 +160,28 @@ scheduler hiccup on top of that — unidentified, and worth a perfetto trace rat
|
||||
cheap in absolute terms, but it is work the frame has no reason to wait for, and the threshold keeps small
|
||||
folders on the simple synchronous path.
|
||||
|
||||
### The same cost, delivered in pieces
|
||||
|
||||
Those two seconds of `readdir` and `stat` cannot be made much cheaper — they are round trips to another
|
||||
process — so the fix is to **draw while they happen**:
|
||||
|
||||
- `StorageSource.listBatches` hands a folder over 200 entries at a time. `AllFilesSource` implements it
|
||||
with `Files.newDirectoryStream`, which is lazy: it pulls `readdir` results as the loop asks for them,
|
||||
where `File.listFiles` materialised the whole folder before returning anything. The default
|
||||
implementation is still a single batch, so the SAF backend and every test fake behave exactly as before.
|
||||
- `BrowserViewModel.load` merges each batch into the list already on screen (`core/SortedMerge`,
|
||||
O(n + k) instead of re-sorting the folder after every batch), so rows arrive in the user's order and the
|
||||
list is never a partly ordered jumble that shuffles into place afterwards.
|
||||
- `loading` stays true until the last batch, which is what the existing progress bar above the list already
|
||||
draws — the feedback that keeps a slow folder from feeling like a frozen one.
|
||||
|
||||
At 200 entries per batch, the first rows should appear after roughly 20 ms of reading plus a frame, against
|
||||
two seconds of empty screen before. **That expectation is not measured yet**: confirming it needs a
|
||||
20 000-file fixture, which takes about half an hour to build through MIUI's storage layer, so the numbers to
|
||||
beat are the ones in the table above and the first thing to do with that fixture is to re-run
|
||||
`openAFolderOfAThousandFiles` and look at the *min* iteration rather than the median — the minimum is what
|
||||
"first rows visible" now governs.
|
||||
|
||||
Building that fixture is the expensive part, not measuring it: 20 000 files took about thirty minutes
|
||||
through MIUI's storage layer (`: >` in a loop), against roughly four minutes of benchmark on top. Worth
|
||||
knowing before anyone re-creates it, and an argument for the fixture being built once and kept.
|
||||
|
||||
Reference in New Issue
Block a user