Thursday 3 November 2022

Thoughts on Backing Up My Calibre Library

I thought it was time to back up my Calibre library that now is 39.8 GB in size and so I dusted off my trusty 128 GB SanDisk and thrust it into the USB 3.0 port on my old 2013 MacBook Pro. From previous experience I knew that this would be a slow process but today I got to thinking about how slow. Figure 1 shows the situation.

Figure 1

The USB 3.0 ports on the MacBook Pro can transfer data at the rate of up to 5Gbps. Now that's 5 gigabits per second. There are 8 bits in a byte and 1024 x 1024 bytes in a megabyte (MB) so that translates to a transfer speed of up to 640 MB/s.  For my file transfers I'm achieving a rate of around 4 MB/s which is 160 times slower than the maximum possible! This seems impossibly slow.

However, if we look online we find the following (see Figure 2):

Figure 2

Even SanDisk only claims a speed of up to 100 MB/s for this particular device and  presumably this is for reading of data and not writing. Furthermore, the thumb drive spends more time checking that writing and so if there are many small files (and I am transferring 34,646 of them), the transfer time will be very much longer than if I were transferring a single 39.8 MB file.

On this particular site, it's claimed that 3.0 USB has a 10~20 MB/s write speed and if that's the case then the 4 MB/s for my worst case scenario (thousands of small files) is not unreasonable. Overall it's not a problem for me because I'm in no hurry and I only carry out the backup every couple of months or so.

I'm also merging the latest library with the older one and this probably adds to the checking time as the new folder (every book has its own folder) has to be compared to the older one. It would probably be quicker just to delete the old library and simply copy the new one across.

An alternative approach would be to make a note of any new books added and then drop their folders into the library backup from time to time. On reflection this makes the most sense and the update would then be almost instantaneous given the small file sizes involved. The current library on my MacBook's SSD and the library backup on my SanDisk will be identical once the transfer is complete. This would be a good time to implement such a scheme. I'll test it out and report back.

Even if I don't make a formal note of my added books, it's easy to identify them by looking at the current library (arranged in chronological order) and noting the dates of books that are newer than the time of my last backup. This could be painful if I let too much time elapse before checking because I will have to locate each book's folder in the database and then copy and paste it across. For a few books it's no problem.

ADDENDUM: November 4th 2020

I've tested the folder copying update method out and it works fine. Quick and painless. Definitely the way to go in future. I downloaded Walter Rodney's "How Europe Underdeveloped Africa" after a recommendation by Jeremy Corbyn shown in this tweet.

The classic work of political, economic, and historical analysis, powerfully introduced by Angela DavisIn his short life, the Guyanese intellectual Walter Rodney emerged as one of the leading thinkers and activists of the anticolonial revolution, leading movements in North America, South America, the African continent, and the Caribbean. In each locale, Rodney found himself a lightning rod for working class Black Power. His deportation catalyzed 20th century Jamaica's most significant rebellion, the 1968 Rodney riots, and his scholarship trained a generation how to think politics at an international scale. In 1980, shortly after founding of the Working People's Alliance in Guyana, the 38-year-old Rodney would be assassinated. In his magnum opus, How Europe Underdeveloped Africa, Rodney incisively argues that grasping "the great divergence" between the west and the rest can only be explained as the exploitation of the latter by the former. This meticulously researched analysis of the abiding repercussions of European colonialism on the continent of Africa has not only informed decades of scholarship and activism, it remains an indispensable study for grasping global inequality today.

No comments:

Post a Comment