At RubyFringe last July, Giles Bowkett gave a presentation that, afterwards, rapidly became the talk of the internet. ‘He had 900 slides,’ they said. ‘He wrote a program that composes music,’ they whispered. ‘He went 2 hours over, but the whole crowd still stuck around to listen.’
The video of the talk recently popped up on InfoQ and if you watch it, as I recently did, you’ll find that though, as usual, some of the chatter was exaggerated it really is a rip-roarer. Giles weaves between relating hi-jinx from his, um, colorful life, saying inspiring things about programming as art and individual creative liberation, and demoing this really cool piece of Ruby for generating MIDI.
Arc-a-what-a-who?
The missing link?
How you play drums in Reason.
It’s the last of those that I want to talk to you about today. Giles’s MIDI generator is called Archaeopteryx and is—in addition to suprisingly hard to spell—a probabilistic step sequencer.
What’s “a probabilistic step sequencer”?
Not an unreasonable question. A sequencer is simply a bit of software that tells the computer when to play certain bits of music, usually via MIDI. A step sequencer, then, is a particular flavor of sequencer that breaks music down into its individual beats (called steps) and allows you to set the status of each one separately. A canonical example is the ReDrum drum sequencer in Reason. The sixteen square buttons across the bottom each correspond to one sixteenth note of the bar. For each drum (represented by the 10 channel strips that make up most of the instrument), you can turn each of those 16 lights on and off to indicate whether the drum should be played on that particular beat. Interfaces like this, in both hardware and soft, are how people “program” drum beats.
So, finally, what about the probabilistic part? Well, this is where things get interesting. As opposed to ReDrum, where you definitively turn each beat of each drum on or off, Archaeopteryx lets you add the element of chance. Instead of just turning each 16th note on or off, you define a probability between 0 (never play) and 1 (always play) and, on each pass through the bar, Arx rolls the metaphorical dice to decide whether or not that note gets played. The result, when programmed correctly, can sound like intentional musical variations on an ongoing pattern or theme. And if you use the MIDI that Arx generates to trigger the kinds of sounds that make up dance music, you’ve got yourself an AI DJ, which is largely how Giles uses it.
Hardware? Hardware!
As the guy with the snare drum-playing robot (also here), this is where I start feeling a certain twitch. That twitch goes in two directions.
Today, I’m going to talk about the In.
I’ve built the first proof of concept for a hardware controller for Archaeopteryx. The rest of this post will be spent explaining how it works and talking about some of the repercussions and areas for future research.
Archaeopteryx is designed for live DJing. It’s organized around the principle that the music can never stop and that all changes you make to it should happen in an orderly, musical fashion. For example, when you edit the files that specify the probabilities for your various MIDI channels, Arx polls your files in sync with the tempo of the music you’re playing and only makes changes at the end of every four measures.
The only downside to Arx as a tool for performing music is that you freaking change the music by editing floating point numbers in arrays in a text file! Needless to say, that is not necessarily the most optimal interface for fluent musical performance. Most musician types like actual physical objects that they can lay their hands on understand with their spatial intuition. Hence, a hardware controller.
What would controlling Arx from hardware mean? Well, at the most primitive level, it would mean setting Arx’s probabilities by manipulating some object in the physical world. I started with a basic knob.
The knob, or more formally, ‘audio taper potentiometer’, is the classic first analog input in physical computing. You plug one of its three leads into ground, one into 5 volts, and the third one into one of the Arduino‘s analog input pins, and you’re good to go.
Once I had the knob plugged in, I wrote a simple program for the Arduino using RAD. The first iteration of the program simply sent the values of the knob back to my computer over serial. I connected to the serial port with screen and watched the numbers twitch. After some observation, I learned that the knobs output values varied from 925 to 1023. I also learned that there was enough jitter that I’d have to introduce a ‘handshake’ to smooth things out.
A ‘handshake’ is a classic technique for getting two devices to communicate with each other smoothly over something like a serial line. The problem is that the output device (the Arduino) may send data faster than the input device (my Mac) can consume it, leading to jitter as some of the intermediary readings are lost. The solution is to let the input device drive the communication. You setup the output device to “only speak when spoken to,” i.e. to only send data when it receives data. Then you have the input device send a bit out every time it’s ready for new data. The result is a smooth conversation.
Taking the handshake into account, here’s the code I came up with for programming the Arduino:
The only possibly surprising bit is the “serial_read” at the end, but that’s just a way of having the Arduino clear the serial buffer so that it’s ready when the computer next tries to communicate with it.
Once I’d gotten the values of the knob into the computer, then what? How did I get it to actually control Arx’s probabilities? Well, as I mentioned earlier, Arx uses a series of Ruby programs to sequence its instruments. Here’s a fragment of an example file that ships with Arx, db_drum_definition.rb:
There they are, the probabilities, right there!
Since this was just a proof of concept, I decided to do the most primitive thing possible: just scale the value coming off the knob into a percentage and set that percentage as the probability for every step of ten MIDI channels—enough to control that whole ReDrum drum kit we looked at earlier. It would be: The Crazy Knob. When turned all the way to the left, Arx would never play, and when turned all the way to the right it would play all ten drums on every sixteenth note. In between it would play random combinations of note with increasing franticness.
And, since this is just a proof of concept, I also decided to go with the simplest delivery method for the data I possibly could: simply modifying the db_drum_definition.rb file in place as if I had edited it by hand. I’m sure there’s a more sophisticated way to do control Arx using some for of it’s public API, but this was the simplest and most brutally effective thing I could think of. I even packaged the whole templating process up in a nice little wrapper I’m calling Templarx so that its hideous guts would be hidden (Templarx is on the GitHub if you want to drop in and make some improvements).
Once that was done, I just had to write a Ruby script that would read the serial data coming in from the Arduino, scale the results into a Float probability, and then call Templarx to re-write Arx’s drum definition file. The result looks a lot like this:
Despite being a little ugly because of how much boilerplate is required to startup a connection with ruby/serialport, this code is pretty straightforward. The only surprising bit is the “sleep 10”, which I had to add because of a detail about the Arduino hardware. The Arduino uses serial over USB to program the device as well as to communicate with the external computer. When you first make a serial connection to the Arduino it goes into a special mode where it waits to see if you’re going to send it data or tell it to reprogram itself. The “sleep 10” is there to give it sufficient time to exit this mode and be ready to listen for normal serial communication. The rest of the code is simply the computer’s side of the handshake and scaling operations described above along with the call to Templarx to write the new probabilities out to the drum definition file where Arx will pick them up on its next cycle.
And…TADA! That’s it. You start up the Ruby program, wait 10 seconds, and start turning the knob. All the way left for quiet. All the way right for max crazy. For maximum results, follow Giles instructions on how to get Arx to drive Reason, plug Arx’s MIDI into ReDrum, and start twiddling the knob.
Or, you can watch me screencast here demoing the results of doing just that:
Ruby Arduino contolling Archaeopteryx from Greg Borenstein on Vimeo.
What’s next?
Well, besides the Out part of the equation (using Arx’s MIDI to control physical hardware), I’ve also started working on some sketches towards a complete hardware controller for Arx. I’m imagining 16 Spectra Symbol Soft Pots to control the probabilities and a selector for which MIDI channel you’d like to control. I don’t know why, but for some reason I picture this project using the awesome old school 7-segment LEDs rather than anything fancy like LCDs.
If you’re a DJ type, or anyone else who’s interested in using Arx in performance—or, better, has already done so—I’d love to hear your thoughts about this design: how it would work for you, what could make it better, what other features it might need, etc.
On the Arduino/physical computing side, I hope this kind of project can be inspiring for beginners. I think it shows how much of a cool effect you can get with even the simplest of introductory circuits if you glue together the right pieces of software to do cool things.
One final piece of awesomeness worth relating is that, while I was working on this, I described it to my friend Devin and he went ahead and used Templarx to whip together an Archaeopteryx controller in Shoes, the amazing minimal cross-platform Ruby GUI toolkit.
Tagged: ruby, arduino, archaeopteryx, midi, computerkraft, music, reason