Convert: A faster heuristic could be an option

[RollingStar]

I'm running beet convert on the whole library. If it ever stops (bad command, runs out of space), I have to start from the top and go through the whole list again, for convert to log "Skipping" [insert track here].

Couldn't we cut down on 5-10 file system checks (5-10 tracks in a typical album) by checking folders only?

Probably right before the confirmation:

beets/beetsplug/convert.py

Line 493 in 9587caf

if not (pretend or opts.yes or ui.input_yn(u"Convert? (Y/n)")):

There could be a -fast switch (default to off/normal/current behavior). If -fast is enabled, prune the items list as such:

Compute the parent folder of every destination file. (Tricky here since convert dests are not guaranteed to have the same path format as source)
Prune dupe folders
List the folders and ask the user
Do the normal filesystem check for each file that survived the pruning (track 1)
If it already exists, skip the folder for convert
If it does not exist, add the entire folder to the pipeline

Obviously, this would miss partial folders. Bonus: once you find the first folder that doesn't exist, go back 1-5 folders and do a full check on them.

The convert pipeline might need to be adjusted to be more sequential. I think it goes by track artist if you don't specify -a. (So Aaron Carter would be close to the top even on a VA release)

---

Alternative method

Add a flexattr "converted" to every converted song in the db with a datetime of the last converted date. Default convert query to WHERE converted IS NULL, but put a switch for -full to check every file for existence.

[wisp3rwind]

I must say that I'm not convinced this is the best course of action, in particular without any certainty about the actual bottleneck here. Aside from the complexity of this heuristic, which you already mention, the use case of "bad command, runs out of space" is precisely when you wouldn't know exactly where to continue and the heuristic would fail (because conversion most likely stopped midway through a folder). It seems to me that to be really sure in that situation that everything is converted would require running the -full variant anyway, or possibly extensive manual investigaiton (making conversion more sequential to ease the latter doesn't seem like a viable option, we do want keep all CPUs busy all the time).

When do you see substantial delay, before or after the confirmation prompt? Before would probably be due to beets' general query slowness and not at all specific to convert, after the confirmation might point at IO performance. Even then, I think the first thing to investigate should be more parallelism by putting the should_transcode check and the actual conversion on separate thread pools, such that the former (IO-bound) isn't blocked by subprocess execution (CPU-bound).