How can I keep a large number of EPUB files, managed in calibre, "polished"

Question

As described in this answer, you can manually polish EPUB files so they are updated with the metadata you added or changed in calibre, without converting the EPUB files.

At least on Linux there is a problem with this functionality if you have many files:

• You cannot select all EPUB files (after searching for format:epub) and start the conversion of them all if there are more than 25-50 files.
• You cannot find which files need polishing and which not, so if you lost track after updating several books, you need to start from scratch.
• You cannot easily run ebook-polish from the commandline on all EPUB files, using find. As files with name ...polished.epub might exists and because the output files cannot be specified and need to be renamed.

Is there a more easy way to polish all and only the EPUB files that need updating?

Answer 1

The short answer is yes, and I have ranted about this in parallelizing EPUB polishing.

---

The problem with some Linux systems (Ubuntu) is that there is a limit to the message popup queue. Since ebook polishing (using calibre) is done quicker than the time a message is displayed, this queue overflows and calibre cannot write messages to it any more, and (on last try at least) calibre did lock up.

If you look in any directory where calibre stores the EPUB, you can see that there is a metadata.opf file that contains the metadata for the corresponding ebook. You have to specify this file as argument for the ebook-polish program together (-o option), together with the corresponding EPUB file.

What you therefore need to do is make list of 'outdated' EPUB files (outdated compared to the metadata.opf files in the same directory), and then call:

ebook-polish -o path/to/metadata.opf path/to/book.epub

and rename the output (path/to/book_polished.epub) to path/to/book.epub.

The script referred to in the above rant does exactly that. As prerequisite you need to have GNU parallel installed¹.

With the script I was able to process 1329 polishes in 3m25. Searching and updating 4 changed files in 2400+ item database with 1600+ books now takes less than 2 seconds². Of course files in the calibre database that are not supported by ebook-publish (such as PDF) are completely skipped.

¹ _{In case you wonder, yes the script, via parallel, calls itself for each file to callebook-polish and rename the output}
² _{Fast enought to always try and polish the EPUB files before I synchronise the selected files with the reader devices}