Years and years ago in a galaxy far
away when dialup was still prominent and simply re-downloading files that got corrupted for some reason or another was a major issue due to traffic caps, fees or just the sheer waste of time involved a tool named ZIDRAV (short for Zorba's Incredible Data Repairer and Verifier) was conceived.
It wasn't pretty, straightforward, user-friendly or well engineered but it worked.
What did it do?
It allowed two people to work together to fix a corrupted file with the least amount of data transfer necessary to do so.
It was a 3 step process
- The owner of the damaged copy would create a checksum-map using ZIDRAV and send it to the owner of an intact
copy.
- The owner of the intact copy would then use the tool to create a patch file that contained only the parts where
the checksum-map of the intact copy differed from that of the damaged one (ie. where the corruption was)
- This patch file was then sent back to the owner of the damaged file and the file could be repaired.
ZIDRAV became analogous with this file repair method and earned a bit of fame and/or notoriety in the process. There was an attempt at resurrecting it several years later and it even made it into some Linux distributions' repositories but eventually the maintainers lost interest and it faded into obscurity.
Superior file-delta solutions exist nowadays but most of them require one party to have a copy of both files at the same time so they just don't serve the same purpose as ZIDRAV did.
Enter the latest Toolsley tool.
Repair damaged files with the help of someone who has as intact copy.
File Surgeon is basically a ZIDRAV-compatible file repair solution (down to the files involved). It can be used as-is or in conjunction with the Windows version of ZIDRAV.
The steps are the same:
Step 1: The owner of the damaged file creates a checksum-map file and then sends it to the owner of the intact file.
Step 2: The owner of the intact file creates a patch file from the checksum-map and the file.
Step 3: The owner of the damaged file applies the patch file
Caveat emptor
The JavaScript API standardization process has not yet produced a workable solution for saving large files from browsers. It's possible to read tens of gigabytes of local data into a browser safely but writing is a hack at best that doesn't work on all browsers and comes with severe limitations where it does. What this means for this tool in particular is:
Step 1, checksum-map creation is virtually unlimited and will always work regardless of size since checksum maps don't grow as large as the limitations.
Step 2, patch creation is limited to 500megabytes of corruption in Chrome, 800megabytes in FireFox, 600 megabytes in IE and the rest is unknown.
Step 3, patch applying has the same limits on total file size for the repaired file.
Workarounds
ZIDRAV for Windows could be used as an alternative for both Steps 2 and 3 if the patch or the file is too large to save from the browser.
Additionally there are two alternatives supported for Step 3.
- The creation of an Xdelta compatible VCDIFF file. Xdelta is one of the more advanced file-delta solutions
mentioned earlier. It's cross-platform and should have no issues applying patches to large files
- The creation of a shell script for *nix and OSX that applies the ZIDRAV patch file without having to use ZIDRAV
or any other tool. It simply uses dd which is included by default on those systems.
Check it out and
let us know what you'd like to see next!