Guide: Backing up crap DVDs
10 October 2015
Let's tangent for a moment and talk about DVDs. The DVD standard was designed for the low-powered embedded hardware of the late 1990s. The copy protection is primitive, and the DVD format is public knowledge by now, making it easy to backup or format shift your DVD collection as is your God-given first sale right (unless you live in the US or anywhere that just signed a trade agreement).
For reasons of Christ Knows Why, some publishers (ignorant of how top-down video piracy generally works) think that the best way to prevent this is by using third-party copy protection schemes. This so-called protection is of the "no True Scotsman" variety; it adds a lot of intentional errors to the DVD that would make most software players (and the authors of the spec) rightfully vomit. But that's ok! Because the discs are designed for a True Hardware DVD Player, one that is infinitely more robust than your PC-based counterparts and will soar over the corrupted blocks like a majestic eagle.
Just to repeat this again, these publishers sell a product with severe mastering defects on purpose and just hope the error correction is really good and you don't notice.
This article will walk you through the five categories of dirty tricks used to limit fair use. Normally you only have to worry about a couple of these; but I had the misfortune to buy a DVD (Chris Morris' excellent Brass Eye) which had ALL DAMN 5. And for the first time, none of the DVD playback software I tried would play the episodes when selected on the menu.
This is the official pitch video for the copy-protection found on this DVD.
I had been officially swindled out of £5.75, so for the record I did make the exact face shown here at 1:07. I mean, at least when a music CD is mastered with defects in a similar way (e.g. Copy Control), the Red Book standard forbids those idiots from stamping the CDDA logomark onto their flat plastic turd. But I guess the DVD Consortium has a much simpler set of rules to qualify for their logo, such as "anyone who pays us $5000".
The categories
- CSS. The region coding protection built into the DVD standard. Each VOB file in VIDEO_TS has a 40-bit descrambling key, which is stored on a hidden area of the disc only accessible by querying the drive firmware. If you're lucky enough to have a RPC1 (region-free) drive, then libdvdcss will have no problem yanking the keys. Nearly all drives are RPC2 (a.k.a. you're allowed 5 region changes and you better like the last one because IT'S PERMANENT BUDDY), and will only let you access the disc keys if the drive region is one of the disc regions. Some DVD drives are manufactured by assholes; not only will they withold the keys, but they'll also throw a ton of IO errors if the region is incorrect!
- Analogue copyprotection. Another pack-in option, this is a flag which causes the analogue video ports on a hardware DVD player to send crazy nonsense during the vertical blanking interval, which will either confuse the shit out of the image gain circuitry in an old VHS deck, or tell a new VHS deck/DVR to stop recording and instead display a patronising message about stealing.
- Corrupted sectors. A popular technique for some protection schemes is to simply master the DVD with intentional cockups in the data sectors on the disc. The thinking behind this is that True Hardware DVD Players are meant to deal with misanthropes who leave their discs face-down on sandpaper and powdered glass, and should therefore glide smoothly (hah!) over any unreadable bits. If you try and copy the VOB files straight from the disc with a file manager, it'll die in the arse once it reaches the first 2048-byte sector that it can't read.
- File table abuse. Like most filesystems, UDF can be dumbed down to the idea of a file table (i.e. the tree structure of directories and files), followed by a large blob containing all the binary data. Each entry in the table has a reference to exactly where, inside the huge blob, a file's data starts and finishes. If you're thinking like a copy-protection engineer, you will immediately see the opportunity to abuse this and make hundreds of decoy files with references to overlapping data, creating a mess that appears several times larger the 9GB limit of a dual-layer DVD.
- Title index mauling. Because they just haven't done enough already, the final place to inflict pain on the user is in the title and chapter structure of the DVD. Hey, the standard lets us have up to 99 titles which can reference any chunk of the DVD video stream, why shouldn't we fill them all up with corrupted copies that terminate 15 minutes in! And those chapter breaks; that sounds like an amazing opportunity to randomly skip to different episodes!
How to get around it
I'm going to go through the steps (in order) I went through to surgically remove the damage. You will need:
- A faulty piece of shit video DVD
- A computer with a good DVD drive running Linux (a bootable Ubuntu USB will work in a pinch)
- libdvdcss2
- ddrescue (apt: gddrescue)
- VLC
- dvdbackup/vobcopy/Handbrake (depending on what you want to do with the DVD image)
- Wine + VobBlanker
If you want instructions on how do this kind of thing using only Windows, you're insane. And you'll have to scour through someplace like Doom9.org because I don't know!
CSS
Try playing your disc in VLC.
- If the picture looks fine, your drive is cooperating with libdvdcss.
- If the picture looks green and corrupted and stuttery, then your drive can access the video data stored on the disc but can't get the CSS keys to descramble it; see the end of this section about how to copy the CSS title keys from a working computer.
- If VLC is throwing errors or a blank screen, and dmesg is full of whining like "Read of scrambled sector without authentication", the firmware on your DVD drive has noticed the disc is the wrong region and is telling you at the IO level to piss off. At this point, you are stuck; you can't get around this without first switching the drive's region or flashing the firmware to be RPC1 (aka. region-free).
Region-free flashing requires Windows; try looking up your drive's model using DiscInfo, then check if either the RPC1 firmware database or the RPC1 forums have an exact match. You can properly ruin your drive if you don't know what you're doing; if you're just starting out, try it on an older drive which you can live without if things go wrong.
I can't remember if I'm just lucky, or if I flashed it once at 3am and can't remember, but my Pioneer BDR-206 is a region free drive 99% of the time. (so far it's failed exactly once; it couldn't read the CSS title keys for the brilliant Mr. Show with Bob and David. I had to grab the keys from a machine with a Region 1 drive)
Anyway, do whatever you have to do to get the disc playing in VLC under Linux. The next step is to open up the .dvdcss folder in your home directory; in here is a copy of the extracted CSS title keys of every disc you've played on the computer up until now. Mine were stored in a folder called BRASS_EYE-2010012910090800-0000000063. This is going to be important later on.
Analogue copyprotection
Not an issue, as all DVD and VOB backup tools disable this flag. Moving on!
Corrupted Sectors
I'm going to invoke the famous "duck test" here; if it looks like a duck, walks like a duck, swims like a duck, has feathers like a duck and quacks like a duck, then it is probably best served crispy with hoisin sauce. We're going to treat our DVD just like we would any other damaged, inches-away-from-death piece of media, and use a recovery tool designed to scrub around all of the bad sectors, which (if the copy-protection peddler is to be believed) aren't part of the video stream. It's a command-line tool called ddrescue, and works roughly like the following:
- Create an image file exactly the same size as the source medium
- Do a quick first pass, skipping over any read errors and rescuing as much good data as possible
- Do a series of bisection passes over the holes, attempting to subdivide the corrupted blocks into smaller and smaller chunks until some data can be read
- Repeat the above step over and over again
$ sudo ddrescue -v -b 2048 -r 5 /dev/sr0 brasseye.iso brasseye.log
2048 is the bytes-per-sector size on a standard DVD, so you can't subdivide smaller than this. /dev/sr0 is the block device for my DVD drive, and I want it to retry the bad parts 5 times before giving up.
Before long, I have a brand new ISO that's a near-perfect copy of the defective DVD. True, there's still 1.3mb worth of holes, but they're filled with zeros and won't throw any IO errors. Sweet, let's toss it at VLC!
But wait, what?! But I copied all of it! Well, most of it! It worked fine before!
Yeah, some of you have already got this. Copying the disc image doesn't copy the CSS keys! Let's take another look inside ~/.dvdcss
Notice how we've now got two sets of disc keys: the ones pulled when I originally played the DVD (timestamped 3:40pm), and a new incomplete set guessed by libdvdcss for that ISO I just tried (timestamped 11:41pm). Do the sensible thing and copy the real keys from the old folder into the new disc folder. Now let's try playing that ISO with VLC again.
Much better. This ISO will now work with any video player or tool that uses libdvdcss to handle descrambling.
File table abuse
If you are processing the DVD with a tool that supports loading ISOs through libdvdcss, then you don't have to worry about this.
Me, I wanted to have a try at scouring that broken title index with VobBlanker, which needs a fully decrypted VIDEO_TS folder to work on. There's a nice little tool called dvdbackup; just run it in mirror mode to decrypt the whole DVD structure to a folder.
$ dvdbackup --mirror --input=brasseye.iso --output=brasseye-decss
Depending on how creative the mastering engineer was, you're going to need a lot of hard-drive space for this. As you can see from the preview icons, this DVD had a lot of straight-out duplicated junk, inflating the size of the mirror to an impossible 18.7 GB. But it's still accurate to the file structure of the original DVD image, and we didn't intend to save it back into DVD format, so not a big deal.
Title index mauling
Popular DVD tools such as dvdbackup, handbrake and vobcopy are usually fine for getting the raw video data out of the VOBs; they just need to know what title number to rip.
The easy method of working around a huge list of broken decoy titles, is to navigate to an episode using the DVD menus, and then write down what title number is being played back (e.g. the "Playback -> Title" menu in VLC)
Unfortunately for me, the Brass Eye disc would only play one episode from the menu. It also had about 90 decoy titles in the index, each with a random length between 27 and 33 minutes, and wired up to random (sometimes unplayable) chunks of video.
I mentioned VobBlanker earlier. This is a Windows tool designed to pick and mix titles from the DVD index, and shows information about how they sit in the VOB filesystem structure.
Here we can take advantage of a few things:
- Regardless of how ruined the indexing is, the video data for a title has to be stored in the VOBs consecutively. Jumping around on the disc too much will leave noticeable seek-related gaps in the footage
- In this case, duplicates made by abusing the file table appeared in consecutive order
We can see looking here that the disc can be divided up into rough 1GB-sized chunks, which correspond to an episode's worth of footage. This image here shows one of the VOBs, with an index full of broken decoy titles as seen in the bottom pane. The 3rd column contains the title number, and the 7th column contains the "duration", as seen in VLC's playback controls (good luck guessing which titles are real by the length!). By contrast, the 8th contains the actual size of the content the title marker points to, with most of them being 0 bytes long! Yes, all of these titles are bollocks.
This image shows the correct title (number 46), which as you can see has a sensible duration and the largest filesize associated with it. Oddly enough these always seemed to appear at the end of the chunk.
Archival
After going through and writing down all the correct title numbers, I had something I could give to Handbrake. And there you have it; after 4 hours of slog, finally I can watch the content I paid for! Yep, can't imagine why anyone would be driven to piracy.