Intro to ROM Hacking: Ghost House
Tue 09 June 2020
A long time between updates! I've been distracted by working on ScummVM's Macromedia Director engine, but it's mostly bugfixing and I couldn't find a good focal point. So on a whim I made this! The previous video took maybe a week, this was a good test of writing/filming/editing in two days.
In this video Scott takes apart "Ghost House" by Sega, and figures out how to remove the obnoxious screen flashing effects by reverse engineering the Z80 code. This contains a full step-by-step explanation of how to use the MAME debugger for hacking ROMs, even if you don't …
Article continues here! KEEP READING, YOU
Let's Reverse: Adventures of Lomax graphics
Wed 25 September 2019
I guess I'm a YouTuber now. Wonder how long I can fight this newfound urge to tase a dead rat.
In this video Scott takes apart "Adventures of Lomax" by Psygnosis and reverse engineers the graphics data. This didn't require taking apart the code, and contains a step-by-step explanation of the black-box analysis process. A good introduction for anyone getting started in reversing games!
Article continues here! KEEP READING, YOU
DDD 2018: Reverse Engineering is Good and also For Everyone
Sat 20 April 2019
Last year I gave my first talk at DDD Perth 2018! It was about reverse engineering, how I think everyone interested should be encouraged to try it, and some useful tips for people starting out. Here is the slide deck.
Apologies that the audio is a bit muffled; the microphone had a recording issue, so instead enjoy some lo-fi camcorder goodness. Thanks again to DDD for having me, and for going the extra mile to accomodate first-time presenters. Extra special thanks to David for salvaging these black box recordings (on a long weekend, no less!).
Article continues here! KEEP READING, YOU
Using your car as a giant joystick for $20
Mon 18 December 2017
DISCLAIMER: Cars can be dangerous. The electrical and mechanical systems of a car are not designed for this use case, and you run the risk of damaging them. This project worked fine for this particular vehicle, but could damage critical systems such as steering and braking in a different model. Do not attempt this with a vehicle that isn't yours, or a vehicle that would leave you with no contingency if it were to break. The author will not be held accountable for any damage caused from following these instructions.
This project is about how to rig the controls of nearly any recent car (in my case a 2007 edition Mazda 3) to act as a giant game controller. In a nutshell; input from the car will be scraped as CANbus messages from a cheap OBD-II adapter, then converted to standard joystick and keyboard events, which will be used to drive a video game projected onto a screen in front of the windshield. No physical modifications to the car are required; the only one I made was to pull the fuses for the headlights so as not to blind the projector screen.
The full source code used for the above demo is available at GitHub. With luck, anyone with entry-level Python experience should be able to adapt this for their car. I think this is a nice practical introduction to car hacking and reverse engineering, without the need to spend a lot on exotic debugging hardware.
Article continues here! KEEP READING, YOU
Show Your Working: Fixing a pile of bugs in MAME
Sun 03 December 2017
Show Your Working is a brand new segment where I write up things I have attempted to fix in open source software. Sometimes it's interesting to debug a problem yourself from first principles, even if the codebase is huge and you don't know anything going in. I will try and explain my thought process as I venture out into the weeds armed only with a butter knife.
The software
If you haven't heard of it, MAME is an emulation project with the aim of supporting basically every arcade machine ever made. To do this, thousands of circuit boards and custom chips have been probed, analysed, decapped with acid, and ultimately written out as code. By adding board ROMs (whole dumps of the memory chips containing the copyrighted program data), MAME can emulate a system by connecting together drivers for all of the chipsets as per the layout of the original board. Recently, the sister project MESS has been merged back into the MAME main codebase, adding ~2200 consoles and home computer systems to the lineup. It is an incredible feat of volunteer-driven engineering.
Unlike other emulators, MAME places accuracy and preservation above all else. All CPU code is interpreted (with some IL speedups via the UML architecture) and runs time-shared in a single thread. Instead of tightly coupling the hardware code with the framework for performance, MAME has an infinitely-rewirable generic module architecture, to encourage reuse of chip drivers across platforms. Imagine MAME as a big box of all the different types of chips, plus some templates that explain what chips to take from the box and how to wire them together to make a system.
Also I was not exaggerating when I said every arcade machine ever made. The MAME source tree has slightly north of 6900 .cpp files, and a princely 2.9 million lines of code (4 million if you count headers!). This is a Big Codebase.
Article continues here! KEEP READING, YOU
Show Your Working: Patching a bug in a 25 year old Sierra game
Sat 23 September 2017
Show Your Working is a brand new segment where I write up things I have attempted to fix in open source software. Sometimes it's interesting to debug a problem yourself from first principles, even if the codebase is huge and you don't know anything going in. I will try and explain my thought process as I venture out into the weeds armed only with a butter knife.
The software
ScummVM is a heroic effort to make a cross-platform engine for hundreds of classic adventure games, making them natively playable on modern systems. Since 2010, ScummVM contains an interpreter originally from the FreeSCI project, and by now does a pretty good job playing games based on the Sierra Creative Interpreter. If you haven't heard of ScummVM, I can't recommend it enough.
It's gotten to the point where ScummVM provides a much better experience than emulating the games in DOS/Windows. Still, given that the engine is reverse engineered without the original code, there's all manner of edge cases and quirks that have to be emulated in order for the games to work correctly, as we'll find out!
Article continues here! KEEP READING, YOU
Show Your Working: Turning off PulseAudio "flat volumes"
Sun 23 July 2017
Show Your Working is a brand new segment where I write up things I have attempted to fix in open source software. Sometimes it's interesting to debug a problem yourself from first principles, even if the codebase is huge and you don't know anything going in. I will try and explain my thought process as I venture out into the weeds armed only with a butter knife.
The software
PulseAudio is the software audio mixer for most desktop Linux distributions. Let's not mince words; PulseAudio has an awful reputation, mostly earned from the botched rollout by Ubuntu and other distributions back in 2008. Long-time Linux users remember these as the dark years where games were unplayable, and everything was punctuated by a loud crackling from the constant buffer underruns. The most egregious problems were caused by the aforementioned shonky buffering, plus a reliance on timing-related ALSA features which had never been used/tested properly for many sound drivers. It didn't help that barely any applications had direct support for the PulseAudio API; there were compatibility shims for ALSA/OSS applications, but they were a crapshoot at best.
But that was a decade ago! We've moved from version 0.9 to version 11.1, a metric ass-ton of effort has gone into making the mixer first rate, most Linux sound libraries are fully Pulse compatible, etc. etc. Surely everything is perfect now, right?
Article continues here! KEEP READING, YOU
Balls to learning how to animate, let's film some parkour!
Sat 27 August 2016
Hallå! Welcome to my first game devlog.
I figured I'd start out talking about graphics, as they are a large source of anxiety for first-time developers. And not without reason! Everyone is a judgy bastard when it comes to how your low-budget game is presented. Oh sure, it's become easier for a solo developer to mask the familiar, rancid stench of programmer art with the equivalent of some pine-scented air freshener, but in practice this is a huge creative tradeoff. It really pays off to have a unique and consistent visual style in your work, even if you're not a proper artist.
Which I'm not. Lord no. What little visual talent I have is spread thinly between drafting/technical drawing (a survival skill picked up working as an engineer) and sandwich-grade graphic design. Oh and I can maybe do a scratchy pencil test of a background at 1/8th the speed of a regular artist. Drawing actual characters and animating them? That's just not going to happen.
Article continues here! KEEP READING, YOU
Dead KORG microKEY, Part 2: Noobface McHamfists Desolders His First SMD
Fri 17 June 2016
Oh God. Oh God. I've been putting this one off, but the stuff has arrived. I can't back down now!
A brief résumé of what happened in Part 1: this great little KORG microKEY MIDI controller stopped powering up for reasons unknown. By wiring it up to a J-Link debug probe, we found that the CPU was still alive and kicking, yet somehow the computer was receiving garbled USB messages. As this keyboard has a built-in USB hub, the stab-in-the-dark diagnosis was that the chip responsible (a Genesys GL850G) had gone bad and needed replacing. Unfortunately, the chip is a 28-pin 10mm*8mm surface mounted design, and our bumbling hero has never before soldered an SMD component, much less reworked a board!
To properly capture the fear of someone doing surface-mount rework for the first time, everything in this piece was written minutes after happening.
Article continues here! KEEP READING, YOU
Audio Analysis, Part 1: Digital Audio
Mon 16 November 2015
Today I'm going to try and explain how digital audio works. Most people have a vague idea about how sound works. "Sound is a wave!", they might say. "Sound bounces off things!", adds another. "Sound increases as you get closer to the stacks!", relates one subject matter expert. "What?", asks someone else? "I SAID SOUND INCREASES AS YOU GET CLOSER TO THE STACKS!" "Sorry I couldn't hear you over my crippling tinnitus! Dear Jesus if only I had known more about sound!"
This will be a really simple primer about digital sound waves and how they contain musical notes. To follow along at home, you'll need the excellent free audio editor Audacity. Download it if you don't have it already, then open it up to an empty window.
Right, so what does a single note "look" like? Let's start with a sine wave. Sine waves have an easily recognisable and pure-sounding tone. In our Audacity window, we'll click Generate -> Tone.
Oh look, they've helpfully picked a Sine wave as the default, with a frequency of 440 Hz and an amplitude of 0.8! (I changed the duration to only 5 seconds, as the default of 30 is a bit too long.) Let's hit OK.
Article continues here! KEEP READING, YOU
J-Link Detective Squad: Dead KORG microKEY-37
Sun 08 November 2015
I love the KORG microKEY 37. It's an excellent entry-level MIDI controller that fits unobtrusively on your desk, great for impromptu jamming when you should be focused on something boring like "tax" or "finding a new house before eviction day". The keys feel pretty soft, as they use rubber domes instead of metal springs, but the velocity response is excellent. I highly recommend it, as it's one of those products which basically lives forever and delivers year after year of good service.
I bring this up because back in about February, my KORG microKEY 37 stopped turning on. The unit would no longer connect via USB; each time it would give up at different points during the initial handshake, with dmesg spewing a number of unhappy messages. Here's one attempt at plugging the device in.
[Sun Sep 27 06:00:56 2015] usb 3-1: new full-speed USB device number 17 using xhci_hcd
[Sun Sep 27 06:00:56 2015] usb 3-1: device descriptor read/64, error -71
[Sun Sep 27 06:00:57 2015] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 16.
[Sun Sep 27 06:00:57 2015] usb 3-1: hub failed to enable device, error -22
[Sun Sep 27 06:00:57 2015] usb 3-1: new full-speed USB device number 18 using xhci_hcd
[Sun Sep 27 06:00:57 2015] usb 3-1: device descriptor read/64, error -71
[Sun Sep 27 06:00:57 2015] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 17.
[Sun Sep 27 06:00:57 2015] usb 3-1: hub failed to enable device, error -22
[Sun Sep 27 06:00:57 2015] usb 3-1: new high-speed USB device number 19 using xhci_hcd
[Sun Sep 27 06:00:57 2015] usb 3-1: Device not responding to setup address.
[Sun Sep 27 06:00:57 2015] usb 3-1: Device not responding to setup address.
[Sun Sep 27 06:00:58 2015] usb 3-1: device not accepting address 19, error -71
[Sun Sep 27 06:00:58 2015] usb 3-1: new full-speed USB device number 20 using xhci_hcd
[Sun Sep 27 06:00:58 2015] usb 3-1: Device not responding to setup address.
[Sun Sep 27 06:00:58 2015] usb 3-1: Device not responding to setup address.
[Sun Sep 27 06:00:58 2015] usb 3-1: device not accepting address 20, error -71
[Sun Sep 27 06:00:58 2015] usb usb3-port1: unable to enumerate USB device
Oh snap! I'm torn on what to do... buying a replacement is doable, but this one was limited edition and a replacement wouldn't be the same cool colour scheme! Maybe there's a clue to what went wrong inside the unit?
Article continues here! KEEP READING, YOU
Introducing Mr. Crowbar! Untold minutes of reverse engineering fun for ages 26-29!
Thu 22 October 2015
I believe that one of the best ways of allowing creativity to overcome limited technical skill is to build upon stuff that already exists. Take game modding; just try and count the number of successful games that started out as a modification of an existing one. Would these developers have gotten anywhere as far with their idea if they couldn't build on an existing engine and assets, not to mention community support? WOULD THEY?
Indeed, game modding is an important springboard for new developers to cut their teeth in a familiar setting and form a good understanding of how games work in the real world. There is, however, one rather large barrier of entry for people to start playing with the innards of their favourite game: tools. Occasionally the engine author will encourage custom content and give their developer tools out for free. But most of the time games are 100% not designed with modding in mind, leaving it up to a tiny number of skilled reverse engineers to write their own tools and share them with the community.
Article continues here! KEEP READING, YOU
Guide: Backing up crap DVDs
Sat 10 October 2015
Let's tangent for a moment and talk about DVDs. The DVD standard was designed for the low-powered embedded hardware of the late 1990s. The copy protection is primitive, and the DVD format is public knowledge by now, making it easy to backup or format shift your DVD collection as is your God-given first sale right (unless you live in the US or anywhere that just signed a trade agreement).
For reasons of Christ Knows Why, some publishers (ignorant of how top-down video piracy generally works) think that the best way to prevent this is by using third-party copy protection schemes. This so-called protection is of the "no True Scotsman" variety; it adds a lot of intentional errors to the DVD that would make most software players (and the authors of the spec) rightfully vomit. But that's ok! Because the discs are designed for a True Hardware DVD Player, one that is infinitely more robust than your PC-based counterparts and will soar over the corrupted blocks like a majestic eagle.
Just to repeat this again, these publishers sell a product with severe mastering defects on purpose and just hope the error correction is really good and you don't notice.
This article will walk you through the five categories of dirty tricks used to limit fair use. Normally you only have to worry about a couple of these; but I had the misfortune to buy a DVD (Chris Morris' excellent Brass Eye) which had ALL DAMN 5. And for the first time, none of the DVD playback software I tried would play the episodes when selected on the menu.