I was put in touch with Christina by Nancy Hechinger, my thesis advisor. With just a brief phone conversation for introduction, Christina was extraordinarily generous with her time. She arranged a meeting for me with Henry Lowood, the curator for History of Science and Technology Collections and Film and Media Collections in the Stanford University Libraries, which has Engelbart’s papers as well as those of many other computer pioneers. She gave me a tour of 333 Ravenswood, the headquarters of SRI, where Engelbart’s Augmentation Research Center had its headquarters when it performed its pioneering work and where the DEI still has its offices today. And last, but definitely not least, she took me along to the grand opening of the Computer History Museum at which she briefly introduced me to her father, which was quite a thrill.

In addition to being a surpassingly gracious guide, Christina was also an incredibly articulate spokesperson for the Augment legacy. Talking with her throughout the day greatly deepened my knowledge of the philosophy underlying Engelbart’s intricate and holistic technosocial system.

In some ways, the single most useful part of the trip — at least for advancing my own thinking about my thesis work — was the Computer History Museum itself. Both its successes and its flaws spoke volumes about how best to present this material: what’s most aesthetically interesting about it and what is most easily lost in a traditional museum approach. I’ll spend the rest of this post trying to articulate these successes and failures in a kind of mini-review of the the Museum with a particular eye towards the lessons it might hold for my project.

Even though the museum has been open in different capacities for years, the unveiling of its current show, Revolution: The First 2000 Years of Computing, inaugurates a new era in its history. This installation is more than a single show. It occupies most of the Museum’s permanent installation space and uses all of the most impressive physical materials from its collection. It also encapsulates the museum’s twin curatorial preoccupations: presenting the physicality of computer technology and breaking up the linear progression of technology into thematic interpretive units. For me, the first two of these ambitions came off much more successfully than the second one.

In contemporary life, computer technology seems to have bifurcated into sensuous products and ineffable information. On the one hand we have iPhones and iPads that get described as “lickable” and become foci for almost orgiastic displays of communal commercial lust. On the other we have “architectures of participation” and social networks that encourage subtle but pervasive transformations in our interrelations. The Computer History Museum does a brilliant job telling the pre-history of the physical charisma currently achieved by technology. Where it fails is in trying to get at the intangible side of this same history: how the networks of people and ideas, the social networks and architectures of influence amongst researchers and engineers, lead to technology that the very structure of our world.

The Computer History Museum shows just how charismatic computer hardware can be. Its exhibits are all based around old machines, machines whose physical presence speaks volumes about the worlds their designers imagined.

Take as a case-in-point the museum’s presentation of SAGE, a US military project for tracking and intercepting Soviet aircraft. SAGE was massively expensive, costing more than the Manhattan Project. The Museum has a series of SAGE terminals as well as this stunning model of the entire SAGE complex that was built as part of its design process:

Every exhibit centers around the physical display of some particular hardware system. The design aesthetics of these systems change and evolve over time, almost like works of sculpture arranged to demonstrate the evolution of some particular style or school.

The Atanasoff-Berry Computer, circa 1942.

An IBM mainframe from the early 60s.

Further, the Museum actually works to restore some of its hardware to working order, like its the PDP-1 in a prominent place near off the main lobby which has been lovingly repaired and even retrofitted with new controls so visitors can play Space War on it themselves.

This use of the hardware serves to defamiliarize computing’s past, reminding us that older computers were not just primitive ancestors of today’s sophisticated machines, but that they actually embodied different social arrangements, different ideas of what technology could be and how it could shape and enhance our lives, even different ideas of what made machines beautiful.

The Computer History Museum’s organizational strategy, on the other hand, is not nearly as effective. Eschewing the obvious linear historical narrative, the Museum opts instead for a series of thematic clusterings: Mastering the Game, Networking and the Web, Input/Output, etc. While the goal of transcending a naive progress narrative is admirable, the result of this particular way of cutting up the apple is to render any sense of historical context or interrelation invisible.

Computing is, maybe more than any other technosocial enterprise, interdisciplinary. Its history cannot be told without touching on government bureaucrats who provided funding, labs that built integrative hardware-software systems, artists who took up cutting edge tools to pursue ancient aesthetic ends, hippies and revolutionaries who fastened on and transformed tools originally invented for military ends.

By splitting up the history of computation into these thematic clusters, the museum renders invisible the figures and movements whose approach was broadest. Figures like JCR Licklider and Bob Taylor at ARPA, whose broad-minded funding strategy profoundly shaped computer research in this country through its most important landmarks in the 60s, their immense contributions nearly disappear when split up in this way.

Similarly, revolutionary researchers like Doug Engelbart and John McCarthy, whose work touched many of the museum’s separate themes — they built input/output devices, they participated in the early stages of ARPAnet, etc. — and who did so out of their own integral visions for what computing could be and mean, their work disappears into a series of brief mentions in isolate exhibits. The actual core of their vision, of Engelbart’s vision of collective intelligence enhancement and bootstrapping, of McCarthy’s vision of Artificial Intelligence, is simply never presented in this schema.

Strangely, the system that best survives the dislocating effect of this curatorial approach is Nolan Bushnell’s Atari. Somehow games are presented as always already integral systems. Or, put another way, they’re not thought of as systems at all, not the sum of a mouse and a networking interface and a display, but unified entertainments that are mostly examined for their pop cultural influence.

While I may be offering criticisms here, I don’t in any way mean to condemn the museum’s work. Even as it stands now, the museum is an impressive achievement. The task being asked of it, to propose a full interpretation a technology which transformed human culture and material life in less than 50 years, is awesome in its scope and ambition. Obviously any initial organization or presentation strategy would necessarily be a first draft of history, to be refined and improved as time goes on and historians emerge with interpretive regimes of greater sophistication that those that currently exist.

As was in impressive evidence the night of the gala opening, while it iss aging, the founding generation of the computer revolution is still very much alive, present, and involved in the creation of this museum. To have already found a fully effective historical perspective in this context would be astonishing.

At this point it must be enough to begin the process: to gather the physical objects, closely observe them, put them near each other, and bring the people together, to see what happens.

History will show up eventually.

PrimeSense demo video of skeleton tracking with the Kinect and their OpenNI middleware

The PrimeSense middleware takes the depth data coming from the Kinect camera and performs a process called “skeletonization”. It detects individual users who are within view of the camera and then tracks the positions of their bodies. All of a sudden, as a programmer, rather than having access to the pixels of a flat image, or even the depth information from the raw Kinect data, you now have a description of your users as a series of joints in space. For many applications, especially gesture-based interfaces and motion capture, this is exactly the data you need to get started.

Unfortunately, when the PrimeSense middleware was released the software reference implementation that went along with it was Windows only. Slowly but surely since then, this situation has improved. PrimeSense has released packages for Linux and OSX and the open source community has started to work towards integrating them into more accessible environments like Processing and Open Frameworks. This process is not yet complete, but it has gotten to the point where and adventurous person can get started. This post documents my first successes doing just that.

The first step is getting all of the dependencies installed. A bit of a challenge at this point, but possible. (Note: this post is based on my experiences of doing this on OS X and will be specific to that.) Tohm Judson’s OpenNI to Max/MSP via OSC tutorial is the best place to start. If you have never installed MacPorts before his directions will go rather smoothly (though there are a lot of steps). If you are unfortunate enough to have installed MacPorts before upgrading to Snow Leopard, you’re in for a bit of a struggle as the MacPorts automatic upgrade path seems to have gotten badly broken with that change. After much frustration I managed to uninstall all the ports that were affected by the upgrade problem (the MacPorts migration page is a good place to start if you’re in a similar situation) and then proceeded through the rest of the steps outlined in Judson’s tutorial.

Judson’s tutorial is based around OSCeleton, a proxy that broadcasts the skeleton data from the Kinect middleware as OSC messages. OSC is a standard format for real time messaging similar to midi and is supported in many languages and platforms, including Processing and Open Frameworks. Once you’ve successfully gotten to the end of Judson’s tutorial, you’ll have OSC messages representing the skeleton data being transmitted and then you can start writing your own code that receives those messages and does whatever you want with the information.

Once I’d gotten everything successfully installed, I ran the OSCeleton Stickmanetic example just to make sure things were working:

This sketch simply uses the skeleton position information in 2D as an obstacle to some particles falling out of the sky with Box 2D for physics. It’s relatively silly, especially the choice of connecting the shoulder joints directly to the head rather than to the neck as seems a lot more intuitive, but it did prove to me that everything was installed and working successfully.

Then, as a basis for my own code I started with the OSCeleton Processing MotionCapture3D example. This is a Processing sketch that reads the incoming OSC messages from OSCeleton, converts them into points in 3D space representing each of the joints of the body and draws a series of spheres at those points.

I wanted to also add lines between each of the joints so, after some experimentation, I used Processing’s beginShape() function and treated each adjacent pair of joints as vertices for lines. In working through this exercise I constructed the following map of how OSCeleton names each joint:

Obviously, I’m only going into detail on the right side of the body, but equivalent nodes are available for the left arm and leg as well. In addition, it’s worth noting that for whatever reason I wasn’t actually seeing any collar, finger, or ankle joints. I don’t know what causes these to not come across, but in my setup they were not appearing in the OSC messages sent by OSCeleton.

Once I’d successfully finished drawing the vertices, I tried my sketch out with my roommate. Lo and behold, you can track two completely separate users no problem.

A couple of notes about this video. Its sluggishness was caused by the screen capture software I used to record it, not the code itself. When not recording, it ran smoothly at a much higher frame rate on my machine. Also, many of the glitches here are caused by the constrained space of my room. The Kinect can obviously only process the parts of your body that it can see. My room is cramped enough that the two of us could barely fit within the Kinect’s field of view simultaneously. Some of the weird glitches you’re seeing here are when individual joints disappear from view and my code draws them as if they were at the top left corner of the screen.

But now that I’ve gotten the skeleton data into a form that I can use, what to do with it? The first thing I thought of was to use it to change the view of this skeleton itself. After all, even though I’m gathering this data in 3D, you’d barely know it from the display you’re seeing here. And most 3D browsing interfaces are incredibly unintuitive and hard to learn, maybe that’s an area of design where full-body gestures could actually be useful.

I added the Obsessive Camera Direction library to my Processing sketch. OCD is the best Processing camera library I know for intuitive control of the viewport. It has slightly more controls than the commonly used PeasyCam, but is less surprising in my experience.

After I had OCD installed, I configured it to always aim at the joint representing the right hand of the detected figure. Then I calculated the distance between the right and left hand and made it so that controlled the zoom. Moving your hands closer together would cause the camera to zoom in, moving them further apart would zoom out. Finally I made it so that raising both hands above your head would rotate the camera around the figure and moving both hands below the hips would rotate the camera around the opposite way.

Here’s what it looked like when I started playing with it:

The code for this is available here: controlling a 3d camera via gestures with kinect in Processing. This video is dramatically improved from the one above because, in the interim, I discovered MovieMaker, a built-in class that makes it incredibly easy to record movie files of a sketch from directly within Processing.

A next obvious experiment to conduct along this path would be to use this interface to navigate around more interesting 3D data, like a pre-existing 3D model. It would be especially cool to use your head to determine the location and angle of a camera within a 3D space to provide navigation and recording of a virtual environment. And then to use the position of your hands to fast forward or rewind various 3D motions being played back within the virtuality.

Another interesting area that I plan to explore soon is creating 3D “hot spots” for interfaces. In other words, mapping particular parts of 3D space to various application controls that can then be triggered by moving different parts of your body into them. Matching these hot spots to actual physical objects or locations within a real room is particularly interesting. Imagine: bringing your left hand near the top of your bookcase turns on a light, putting your right hand there turns it back off, etc. The possibilities are endless.

My sculpture, “LO: October 29, 1969”, memorializes this moment. It uses laser-cut plexiglass, OLED screens, dollhouse furniture, composited educational films, and Baroque theatrical lighting effects to reenact the birth of our now ubiquitous online world.

LO detail. Node 1: The UCLA Network Management Center

It was a modest moment, lacking ceremony, this first ARPANet transmission. The UCLA and Stanford teams connected their IMPs (Interface Message Processors) and the UCLA team begin the process of logging into Augment’s NLS collaboration system. The two teams were in phone contact with each other; as the UCLA team typed each letter of the initial login command (L – O – G – I – N), they verbally confirmed its arrival in the north.

LO detail. Node 2: The Stanford Augmentation Research Center

However, something, as it always does in software tests, went wrong. When the UCLA team typed the “G”, the Augment system attempted to auto-complete the command to “LOGIN”, sending back too many characters down the ARPANet connection, causing the UCLA system to crash, which then brought down the Augment system in turn.

‘LO’: October 29, 1969 on Vimeo

This moment resonates with me as an origin narrative for a number of reasons. First, as a maker of interactive sculptures that rely on touchy, error-prone technology, I can identify closely with the bug that brought down the system on its first run. I’ve experienced exactly parallel problems in getting hardware projects to communicate with software. I find the presence of such a simple “beginner’s bug” at this epochal moment personally touching.

Secondly, and more importantly, the contrast between the cultures of the two labs on either side of this first Internet connection dramatizes a tension in technocultural development that persists and continues to shape our world today. Namely, the tension between the military industrial complex that funded the personal computing and networking technologies at the heart of our contemporary world and the counterculture which gave them their meaning.

Both the UCLA and Stanford labs received much of their funding from ARPA, the Defense Department’s Advanced Research Projects Agency. Though ARPA’s leadership interpreted its mandate broadly and creatively, their fundamental goals were oriented towards fighting and winning the Cold War. In fact because of this association computer technology was specifically picked out by anti-war protests as an avatar of the establishment’s warmongering. Punch cards were worn in anti-draft protests, students hoping to transfer their inscription of Do Not Fold, Bend, Mutilate or Spindle away from the cards and onto themselves.

However, by 1969 in the Bay Area around Stanford, the counterculture had strongly taken hold. And the Augment lab was not isolated from this change. Doug Engelbart, the lab’s visionary founder, and other of its lead engineers used LSD as part of their design process. The hippie encounter group EST was involved in the management of the lab. Leading countercultural figures, such as Stewart Brand and Ken Kesey, were frequent visitors.

Instead of seeing the computer as simply a tool of “The Man”, the Augment community began to imagine the computer as a consciousness expanding experience similar to other countercultural tools such as psychedelic drugs and communal living.

It is out of the synthesis of these two influences that the modern personal computer and internet cultures emerged: cutting-edge technology used towards the ends of advancing human communication, collaboration, creativity, and community. And it was at Doug Engelbart’s Augment lab that this synthesis first emerged. For the first time, rather than treating its users as information to be processed or cards to be punched, the Engelbart’s system imagined them as countercultural individual striving for expanded consciousness.

While the comparison is not entirely fair to the UCLA lab, LO attempts to emphasize this divide through the material in which each of the labs is rendered. The UCLA lab comes to represent the existing fully military industrial computing approach. That full side of the sculpture is manufactured out of laser-cut white plexiglass. Everything is clean, symmetrical, inhuman.

LO detail. The UCLA Network Management Center.

Further, for the figures in this half of the sculpture, I composited the heads of Vint Cerf and Lenard Kleinrock, two important figures in the creation of the ARPANet, onto a prmotional film from the 1960s explaining the technical workings of timesharing. The footage is black and white and the characters are dressed rather conservatively in suits, ties, and bow ties. I then projected that footage onto a scrim within the sculpture so the figures would appear to be floating within the scene, a technique derived from similar tricks used in Baroque theatre and magic lantern shows from the 18th and 19th centuries.

The Augment side of the sculpture, by contrast, is built of modified doll furniture and props as well as hand-fabricated pieces. It is messy, colorful, and lived-in. I based its appearance as closely as I could on actual photographs taken of the Augment lab in the 60s, archived on Henry Lowood’s terrific MouseSite.

The animation on this side, meant to represent Bill English, one of Engelbart’s chief assistants in charge of hardware, was extracted from a 1968 film imagining the different ways men and women will use technology in “the future”. It has the saturated color palette much associated with both that time and with psychedelic art of the period.

Here is a video of both half of the pieces together as installed at the 2010 ITP Winter Show. You can also hear me hoarsely explaining the piece to a visitor, articulating some of the ideas I have discussed here:

‘LO’: October 29, 1969 detail on Vimeo

To close, a few final technical notes on the piece’s execution. The two sculptures were designed to be halves of a square box. Inside of each one is an Arduino controlling the OLED screens in order to synchronize the display of the characters to represent the transmission of data from UCLA to stanford (see the first video I posted above). The OLED screens, while producing exactly the appearance that I wanted, turned out to be extremely fragile and expensive.

The Arduino in the UCLA side also communicated via USB with a Processing sketch that drove the videos projected onto both sculptures. Synchronization was achieved by having the Processing sketch send a serial message to the UCLA Arduino which would alter its OLED screen and then pass the message on to the Stanford Arduino, hence keeping the video and both of the sculptures in sync. Getting the Processing sketch to successfully play back both pieces of video without crashing was one of the most difficult technical obstacles in creating this project. And, in future installations, I will likely eliminate that component as the timing between the video and the OLED screens does not need to be as precise as I initially thought and using simple PICO projectors driven off of SD cards would allow me to eliminate the computer altogether, dramatically simplifying the setup and robustness of the project.

More photographs of the piece under construction (including details of all of the laser cut components) are available in the LO: October 29, 1969 flickr set.

I’ll be showing this piece beginning February 11 at Ventana244 Gallery in Williamsburg. More details about that show here.

LO: October 29, 1969 installed at the 2010 NYU Winter Show

Since the first piece will inherently need to stand without the others, I needed to select a scene from Engelbart’s story that’s capable of making sense on its own without the context of the larger arc. After some deliberation, I chose the first transmission of data on the internet between the Augment team at Stanford and UCLA on October 29, 1969, scene 10 in the full story, near the end of my second act:

After some sketching in which I imagined the piece as consisting of two-halves of a divided cube, one containing the UCLA side (all white, smooth, and institutional with men-in-suits leaning over massive mainframes) and one the Augment side (funky and cluttered furniture in lurid 60s colors, unkempt hippie-hackers sitting at the small familiar-looking NLS system). Each half would have animated projected figures moving around and small screens enacting the transmission of data between the two halves. The two halves would be connected by a series of miniature telephone polls that would carry the actual connecting cables that would coordinate the two sculptures.

I got started by building a cardboard prototype of half of the piece and then experimenting with projecting onto a scrim suspended in it, using LED lighting to blast out the parts of the image that make it through the scrim onto the walls and furniture:

Here’s a video of the prototype that makes it a little clearer how it works:

First ARPANet transmission: UCLA set prototype from Greg Borenstein on Vimeo.

While I learned a lot from this prototype, I want the final version of the UCLA half to have a slick, shiny finish with the classic coldness of marble, but achieved with laser-cut plexiglass. But I also want to be able to include a level of “gizmology” detail that gives a feeling of the texture of the real environment, for example here’s a photo of the model of Sigma-7 mainframe that was in use at UCLA at the time of the transmission (and is modeled in cardboard in my prototype):

But how to achieve this level of detail while still retaining the clean classic white plastic look I want for the whole piece?

After extensive discussions with ITPers proficient in the use of the laser cutter, I decided to test out using the laser to raster etch the plexi to create a kind of bas relief to represent the Sigma-7’s panels. After designing a fragment of the panel in Illustrator, I sat with ITP resident Paul Rothman and ran a series of tests with different laser settings. Here’s what the results looked like:

(Check out the larger size on Flickr)

While I was skeptical of this etching approach at first, even the first test (on the very left) was quite encouraging. Though shallow and hard to see from far away, the details were definitely legible. So, we proceeded with more tests, constantly tweaking the laser settings to achieve a deeper etch and more definition between the three depths we were cutting.

Here’s a table of the settings we used for each iteration, moving from left to right in the sample illustrated above (Layer 1 is the deepest cut, layer 2 is in the middle, and layer 3 is the shallowest):

#laser-settings tr td {
padding: 5px 7px;
text-align: left;
}

In the end, I was very happy with how the relief turned out. As the cut got deeper, shadows started to emerge that made the details readable even at a pretty good range and the parts actually got to be even somewhat close to their real scale.

I’m currently proceeding towards designing out the rest of the hardware details for cutting and assembly. I’ll post more pictures of the final pieces as they come together.

First the “artist statement”. This is meant to communicate what drives my work in as brief and clear a format as possible. It’s what you would find on a one-page takeaway sheet in a gallery after seeing my work:

My work explores the use of special effects as an artistic medium. I am fascinated by how special effects techniques cross the boundary between images and the physical objects that make them: miniatures, animatronics, motion control photography, 3D rendering, physical fabrication.



I frequently use this medium to examine the cultural history of technology, a history that is intimately tied up with the evolution of these techniques themselves through their common roots in an encounter between the military industrial complex and the counterculture at midcentury.

Next is the “project proposal” for my thesis. This will be increasingly fleshed out in the next few weeks as we reach the end of the thesis preparation process and start work on the actual thesis. It includes some of the background that drives my thesis ideas and the beginnings of an outline for how I will achieve them. Here it is as it currently stands (I’ll probably post a revised draft in the next few weeks when it is officially “completed”):

The mid-century American techno-cultural movement which created the personal computer and internet industries is beginning to enter history. As the internet grows towards ubiquity, the ideas, artifacts, and culture of the small group of people involved in this movement become massively influential on the larger society.

It is time for an art that addresses that history, an art that will tell its story, explore its material forms, and investigate its way of seeing the world. This art should use tools of representation appropriate to the mid-century technological milieu as well as contemporary tools sympathetic to its aesthetic and descended from its ideas.

Both the personal technology that surrounds our daily lives and the special effects tradition of image making arose from an encounter between the military industrial complex and the counterculture at midcentury. Hence the techniques and methods appropriate to representing this material will derive from special effects movie-making

These techniques/media include:

• special effects
• interactive electronics
• networking
• rapid prototyping and fabrication
• motion control
• miniature photography
• 3D modeling

Further, I plan to approach producing the project as if it was itself a special effects-laden hollywood blockbuster, dividing the process into pre-production, production, and post-production phases. This methodology will force me to focus on the structure, clarity, and strength of the storytelling elements as well as the technological developments. This semester I’ve made great strides in the pre-production process; some of what’s listed below in that area, I’ve already written about here before, clearly.

Pre-production:

• story work, character work, set design, and effects proof-of-concept
• much of this work has begun this semester but it will continue for the first few weeks of next semester
• this process will end with a full production schedule

Production:

• building the sets and materials for a series of sculptures (3-6)
• organized into 1-2 week chunks based on medium and material
• producing whatever video and digital content is necessary for the pieces

Post-production:

• combining the materials into final pieces
• editing and refinement
• finish and details
• last 4-6 weeks of the semester

Next is the bibliography. Some of this is reading I’ve already done, some is stuff I intend to read over break and during the actual thesis process. I’ve organized it by area: Computer History, Art, Special Effects.

Computer History:

• What the Doormouse Said by John Markoff
• The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal by M. Mitchell Waldrop
• Where Wizards Stay Up Late: The Origins of the Internet by Katie Hafner
• The Engelbart Hypothesis: Dialogs with Douglas by Valerie Landau
• From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism by Fred Turner
• Spacewar: Fanatic Life and Symbolic Death Among the Computer Bums by Stewart Brand
• Computing in the Middle Ages: A View From the Trenches 1955-1983 by Severo Ornstein
• Creative Capital: Georges Doriot and the Birth of Venture Capital by Spencer E. Ante
• Outrageous Betrayal: The Real Story of Werner Erhard from Est to Exile by Steven Pressman
• Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age by Michael A. Hiltzik
• Hackers: Heroes of the Computer Revolution by Steven Levy
• Core Memory: A Visual Survey of Vintage Computers by John Alderman
• On the Way to the Web: The Secret History of the Internet and its Founders by Michael A. Banks

Art

• Star Wars: A New Heap by John Powers

Robert Smithson: The Collected Writings

Archaeologies of the Future: The Desire Called Utopia and Other Science Fictions (Poetics of Social Forms) by Fredric Jameson

Special Effects

• The Vatican to Vegas: The History of Special Effects by Norman M. Klein
• Industrial Light & Magic: The Art of Special Effects by Thomas G. Smith and George Lucas
• The Making of Star Wars by JW Rinzler
• The Making of Empire Strikes Back by JW Rinzler
• Special Effects: An Oral History — Interviews with 37 Masters Spanning 100 Years by Pascal Pinteau

Another exercise from the thesis project was to list keywords we associate with our projects. This is part of the process of examining theaesthetic and intellectua terrain immediately surrounding our project. Here were mine:

• movies
• special effects
• phantasmagoria
• pepper’s ghost
• technology
• history
• history painting
• miniature
• monument
• narrative
• counterculture
• psychedelia
• projection

And, finally, here’s another link dump, this time of links to artists, films, and other aesthetic work relevant to my project. The idea here was to find the “siblings” to our practice: people and groups who make work that has a family resemblance and maybe even a bit of sibling rivalry.

• Jennifer and Kevin McCoy
• Gregory Barsamanian
• David Opdyke
• The Museum of Jurassic Technology
• Rachel Whiteread
• Arthur Ganson
• Lord of the Rings and King Kong making-ofs
• Georges Melies
• Yin Xiuzhen
• Primer
• Filipe Pinto Soares
• La Jetee
• John Powers
• Ray Harryhausen
• Ralph McQuarrie
• Charles Simonds
• Bodys Isek Kingelez
• Jonathan Schipper
• John Klima
• Mariela Neudecker
• Drop City
• Pierre Huyghe
• Anthony Goicolea
• Graeme Patterson
• Thomas Doyle
• Tim Noble & Sue Webster
• Captain Scarlet and the Mysterons
• Gregory Crewdson’s Natural Wonder Series
• Filip Vervaet
• Tetsumi Kudo

I imagined a table-top monument: a square box of shiny ipod-white plastic containing a rolling Silicon Valley vista of blue foam covered by a thin veneer of fake grass. At the center of this would sit the temple-like SRI building in a rough 3d-printed white, like a post-modern Parthenon. And, hovering over this, would be a translucent Engelbart, a pepper’s ghost reproduction of the classic portrait: pointing forward to the future with the dangling cord of some unknown machine augmentation dangling out of his ear.

I even made a furious sketch of the idea on a flight out west, complete with classical pedestal:

When I began talking about my project around ITP this semester, I used this image as a starting point, taking off from it to explain the larger idea of making art about the history of the personal computer. I found that it worked quite badly for this purpose. In order to get fellow students and professors to understand any of the importance I attached to this image — why I’d want to make such a seemingly strange and specific monument — I found myself gropingly telling the story of Augment as a linear narrative. It was a monument that only had meaning if you already knew the details of the story and why they mattered. Hence all the work I’ve been doing (and documenting on this blog) to figure out how to tell Doug Engelbart’s story in a way that is engaging and powerful even for the uninitiated.

Last week, in my Materials and Building Strategies class, we presented prototypes for larger projects we intend to work on for the rest of the semester. Even though I now realize that this piece about Engelbart haunting SRI is not a good starting point viewers of my project, I’ve yet to decide which scene is, exactly. Should it be the scene of Engelbart and English assembling the first mouse prototype? Or the scene of SRI participating in the first ARPAnet transmission? Or the scene of Stewart Brand and Ken Kesey visiting Augment with the Merry Pranksters bus?

Due to this indecision, I went ahead and prototyped this original idea. I figured that a lot of its elements will end up being included in whichever scene I do decide to take on.

Engelbart Haunting 333 Ravenswood prototype from Greg Borenstein on Vimeo.

I built the box out of foam core. The base is architectural modeler’s grass over blue foam. The building is blue foam painted white and the pepper’s ghost is just a piece of plexi attached at a 45 degree angle to a cheap digital picture frame.

This week I explored those questions by creating a “mood board” — a collage of existing visual elements that give a sense of the colors and textures that I want the final piece to have.

Checkout the larger size on Flickr.

As I worked on the collage, three clusters of images took shape: color snapshots from the 60s, special effects making-of imagery, especially blue screen work, and white architectural imagery, especially that with a Greek, neo-classical, or minimalist look.

The 60s snapshots (many of which are of actual period Augment activities) have a saturated color palette and a kind of soft focus that is core to how I visually imagine that style. I imagine this look being especially prevalent for the psychedelic and countercultural material in acts 2 and 3.

The blue screen and special effects imagery comes out of my sense of the relationship between the technical form of image making that make up the modern “cinema of attractions” (computer graphics, motion control, compositing) and the early personal computer world of Augment. Behind-the-scenes process shots from that world feel like a contemporary counterpart to equivalent 60s scenes of bearded men in blue jeans playing primitive computer games and building the first robots.

Finally the classical/minimalist white architectural imagery has a relationship to the material that’s a little bit harder to define than the other two clusters. Partially it comes out of my sense that what I’m trying to build is a kind of monument. Art historically, monumental sculpture (and, ironically, the ground-based minimalist sculpture that most sought to resist it) has tended to have a strong relationship to architecture via the formats of the plinth and the pedestal. Further, the austere white in which classical architecture and sculpture comes down to us (however distant that is from how it was originally constructed) has developed deep associations with the arts of memorialization and monument building. I envision this portion of the aesthetic spectrum as, at least in part, relating to the framing and presentation of the work.

To try to start answering these questions, I’ve been working on expanding the outline into a series of storyboards. In this post, I’ll go through the storyboards for each of the acts and mention a few things that I learned while working on them.

Act 1:

As a reminder, here’s the written outline for this act:

• Engelbart ships out on VJ day. The end of the war is declared as his ship pulls away from the dock
• Engelbart is bored in a grass hut in the Phillipines on radio duty. Someone drops off a copy of the Atlantic Monthly with Vannevar Bush’s As We May Think in it. He dreams of Memexes.
• Engelbart, returned to California, happily married, and bored, pulls over on the side of the road suddenly and realizes he’s achieved all of his dreams. He has a vision of using computers to help people better understand their complex world.
• Now, having finished grad school, and working as a junior researcher at the Stanford Research Institute, Engelbart meets Bill English, a well-liked fellow researcher who has a knack for actually implementing things and getting people to follow him. Doug tells Bill about his Augment dream.
• Bob Taylor from NASA hears about Doug’s Augment idea and decides to support it with its first real funding.
• Doug gets a lab for the first time.

Working on this act, I was surprised at which scenes were easy and which were difficult. For example, scene #5 here, Bob Taylor giving Engelbart his first federal funding seemed incredibly difficult and dry to me in theory but in the process of drawing it, something about having the Washington Monument out the window and figuring out Taylor’s square military haircut as the core of his caricature really made it come together. Similarly, the moment of Engelbart and English meeting at SRI seemed overly abstract in spec, but was crystalized for me when I saw the photo on SRI’s page about the history of its magnetic logic lab.

On the other hand, a scene which had been especially vivid in my imagination, Engelbart’s vision by the side of the road, doesn’t look like much when I draw it out. I’m going to have to think hard about how to bring that scene to life.

Act 2:

• Engelbart has 2 LSD experiences at the International Foundation for Advanced Study. In the first one he is by himself and stares catatonic at a wall for 8 hours. In the second one, he’s with other engineers and co-workers and has a vision of a “tinkle toy” for helping potty train young boys.
• Engelbart and English build the first mouse prototype and it performs extremely well in early tests of input devices
• Augment people visit the Stanford Artificial Intelligence Labratory and play Space War
• Augment participates in sending the first ARPAnet communication to UCLA, the command “LOGIN”, which causes the system to crash after “LOG”
• Stewart Brand helps Augment prepare the Mother of All Demos presentation, their public triumph

One of the challenges here is going to be to figure out how to bring out the counterculture thread that starts in this act. Opening it with Engelbart’s LSD vision gets us off in the right direction (and I love the image of the toilet vision), but it’s something of a challenge to keep that in focus in the other scenes. One important opportunity for that is going to be in the costume and set dressing of the scene at SAIL: the engineers are going to need to start looking scruffy and the environment should start having some psychedelic art/coloration going on. A key to this as well, I think, is going to be shifting the color palette away from the austere almost black-and-white of the 50s towards the washed out rich colors we associate with the 60s. That transition should start here and really reach completion in Act 3.

Also, the idea I came up with for how to tell the story of the first ARPAnet message — combining the UCLA and Augment sides of the scene so that they almost appear to be in the same room — will be a bit of a challenge to realize. How do you represent the lightning bolt/scene divide? Also who are the people on the UCLA side and how do we know they’re not just some Augment extras we haven’t encountered before?

Act 3:

• Stewart Brand brings Ken Kesey to the lab to use the Augment technology and he says “It’s the next thing after acid.”
• Augment researchers visit Lama, a hippie commune in Taos, New Mexico with Brand and live temporarily amongst the Bucky domes and hippies.
• Augment researchers become obsessed with Est, a pseudo-psychological cult of ‘interventions’; members of the lab are caught doing a drug-fueled, computer-enhanced encounter session while Engelbart is giving a tour to pentagon funders
• Bob Taylor recruits Bill English away to Xerox PARC; he’s the first of many top Augment staff to leave.
• JCR Licklider returns to ARPA and cuts off Augment’s funding
• Engelbart is left alone, using NLS by himself in a closed-down Augment lab.

Now here we have lots of new elements: exterior sets for the first time with the Lama visit and English’s recruitment at PARC. Also there are some interesting echoes of early scenes, for example the est intervention session makes an interesting parallel with the SAIL visit from Act 2. Also, and this was not something I realized until I drew the storyboard, the last two scenes of this act (Licklider cutting off Augment’s funding and, Engelbart alone in a cleared-out Augment) echo the last two scenes of Act 1 in a way that I think is really satisfying. You kind of get the whole arc of the story right there: he’s back where he started, but now the personal computer is a real thing sitting on his desk.

Another question for all of these acts is: what role can a moving camera play? If this was a traditional hollywood film, you’d want to vary the camera approach from scene to scene so that the film took on a more dynamic exciting feeling during important action sequences. What’s the equivalent of that here? It seems important to use some motion to break up the static quality things could fall into, especially as the arc of the story moves from the 50s into the 60s countercultural parts, but how to do it?

One big positive thing that’s come out of making these storyboards is that they’ve gotten me thinking about the kinds of things I’m going to have to build or represent in each scene. I’ve got the beginnings of a catalog of effects, props, people, and locations going in my head now.

In this post, I’d like to talk about one component of that process: the character design. Unlike most blockbusters, I’m starting with real people and using media and objects instead of actors to represent them. Hence character design is extremely important. Will the viewer immediately be able to recognize each of my four or five main characters across different pieces that render them in different forms and possibly different media?

In order to ensure that they will, I’m conducting a series of visual studies of my main characters designed to reduce them to caricatures. For each person who figures prominently in the story, I’m making study drawings of them from historical photographs with an eye towards the question: what is the single defining visual characteristic needed to represent them? How simply can I indicate their visual identity.

Here are a few examples of what I’ve come up with so far.

Bill English. Engelbart’s right-hand man, the Chief Engineer and Assistant Director of Augment. His two defining features seem to be his big dark glasses and his round domelike forehead, emphasized by his slightly receding haircut.

Doug Engelbart. Founder of Augment. Our hero. This is a little bit of a tough one as in many ways Engelbart was the typical suit-and-tie engineer-type. But after making this drawing and a few others, I focused in on the hair and the head shape. His hair has this signature pompadour flow to it and striking shocks of gray and his whole head has a very solid rectangular shape with a very strong chin.

Now, here’s an example of how these two caricatures in action. The following is a frame from my storyboard of the scenes in Act 1 (about which more in the future):

Hopefully, even with just those few extremely messy strokes, the presence of the hair and the glasses let you know that what you’re seeing is Engelbart and English looking through a door into an empty room.

Let’s take a look at two of our secondary characters as well. In many ways these are harder as they are likely to get less “screen” time and hence need to be instantly recognizable.

This first one is John McCarthy, the head of the Stanford Artificial Intelligence Lab, which had a kind of sibling rivalry relationship with Augment. Thankfully, McCarthy is a caricaturist’s dream: giant halo of frizzy hair and beard, big black glasses and prominent nose. The glasses alone wouldn’t do it for us as we’ve already got glasses as the primary element of English’s caricature, but with all of McCarthy’s frizzy hair poking out in every direction they should be confused.

This last “character” is a little different, but I think it plays an important enough role in the story to actually be worth considering as a character rather than a prop or an effect.

The original mouse prototype. English built it out of wood and two potentiometers from a sketch by Engelbart; its invention was one of the most famous achievements of the lab. As a caricature, I think the things most important to emphasize are its large size compared with today’s mice, its primitive construction, and the fact that it’s made out of wood.

Next time, I’ll have an example of using these caricatures: a basic storyboard from the outline I presented last time.

So, what group of people, what myth to tell, and how to tell it? As an answer, here’s the start of a manifesto or mission statement:

The mid-century American techno-cultural movement which created the personal computer and internet industries is beginning to enter history. As the internet grows towards ubiquity, the ideas, artifacts, and culture of the small group of people involved in this movement become massively influential on the larger society. It is time for an art that addresses that history, an art that will tell its story, explore its material forms, and investigate its way of seeing the world. This art should use tools of representation appropriate to the mid-century technological milieu as well as contemporary tools sympathetic to its aesthetic and descended from its ideas. These include special effects (miniature photography, compositing, etc.), rapid prototyping and fabrication, motion control, computer graphics.

Who created the personal computer? What were they trying to achieve in doing so and what became of them? It is amazing that, in contrast to other technologies with equal impact on our lives, we have no shared myth to answer these questions. What’s personal computer equivalent to the story of Ben Franklin with his kite or Alexander Graham Bell with his wire?

The answer lies in the biography of Doug Engelbart, a World War II radio operator and early computer engineer who is the first person to have conceived of the computer as a device that could be used by individuals to aid in their own personal work for memory recall, information organization, communication, etc.

Sculpting the rich facts of Engelbart’s life into a story that has the shape and impact of myth is a challenging prospect. As a way into that problem, I’ve been employing the process used by screenwriters in developing movie scripts: trying to boil down the events of Engelbart’s life into a three act outline that focuses on the ways in which Engelbart and a small number of supporting characters around him were changed by the events that took place. As an artistic choice, emulating the “pre-production” process of a major motion picture seems especially appropriate for this subject matter as I intend to use many of the techniques and materials of movie special effects to create the images and objects that will be the final result. As I understand it now, the final result will not likely be a big budget blockbuster, but I’m treating the storytelling and design process as if that was what I was working towards.

So, here’s my current draft outline for a three act story that could be called something like “Augment: The Rise and Fall of Doug Engelbart.”

Act 1

• Engelbart ships out on VJ day. The end of the war is declared as his ship pulls away from the dock
• Engelbart is bored in a grass hut in the Phillipines on radio duty. Someone drops off a copy of the Atlantic Monthly with Vannevar Bush’s As We May Think in it. He dreams of Memexes.
• Engelbart, returned to California, happily married, and bored, pulls over on the side of the road suddenly and realizes he’s achieved all of his dreams. He has a vision of using computers to help people better understand their complex world.
• Now, having finished grad school, and working as a junior researcher at the Stanford Research Institute, Engelbart meets Bill English, a well-liked fellow researcher who has a knack for actually implementing things and getting people to follow him. Doug tells Bill about his Augment dream.
• Bob Taylor from NASA hears about Doug’s Augment idea and decides to support it with its first real funding. Doug gets a lab for the first time.

Act 2

• Engelbart has 2 LSD experiences at the International Foundation for Advanced Study. In the first one he is by himself and stares catatonic at a wall for 8 hours. In the second one, he’s with other engineers and co-workers and has a vision of a “tinkle toy” for helping potty train young boys.
• Engelbart and English build the first mouse prototype and it performs extremely well in early tests of input devices
• Augment people visit the Stanford Artificial Intelligence Labratory and play Space War
• Augment participates in sending the first ARPAnet communication to UCLA, the command “LOGIN”, which causes the system to crash after “LOG”
• Stewart Brand helps Augment prepare the Mother of All Demos presentation, their public triumph

Act 3

• Stewart Brand brings Ken Kesey to the lab to use the Augment technology and he says “It’s the next thing after acid.”
• Augment researchers visit Lama, a hippie commune in Taos, New Mexico with Brand and live temporarily amongst the Bucky domes and hippies.
• Augment researchers become obsessed with Est, a pseudo-psychological cult of ‘interventions’; members of the lab are caught doing a drug-fueled, computer-enhanced encounter session while Engelbart is giving a tour to pentagon funders
• Bob Taylor recruits Bill English away to Xerox PARC; he’s the first of many top Augment staff to leave.
• JCR Licklider returns to ARPA and cuts off Augment’s funding
• Engelbart tries to convince Bob Taylor to bring all of Augment over to PARC, but Taylor refuses
• Engelbart is left alone, using NLS by himself in a closed-down Augment lab.

Now, with the story in mind, let’s return to the question of what group of people mind find this myth important to their self-definition and identity. (Note: every point in that outline is true insofar as it supported by one or more party in the current historical record of this period as it is available to us; it is a myth solely in the sense that I’ve tried to reduce a complex and multifaceted story into a linear one with a clear and iconic shape.)

At the broadest level, Engelbart’s story is important to all of us who live surrounded by personal computers, smart phones, social networking sites, and the other media through which we conduct contemporary life, all of which have their root in this encounter between the military industrial complex and the counterculture in the bay area in the 1960s.

At a deeper level, this myth is vitally important to people who work in these fields: making websites, designing and building computers and the next generation of technology. It puts their work in context and gives it a sense of importance and tradition. From the founders of Google, Apple, and Microsoft to the young engineers and designers working on Twitter and Facebook, this story is at the heart of the battle over the meaning of what they do.

Even more specifically, this story is a matter of active concern to the older generation of technologists who were actually part of it and who are now entering the phase of life where they find themselves concerned with their place in history. Many of them have specifically dedicated their energies to ensuring that aspects of the history of technology, from Andy Herzfeld’s work at folklore.org documenting the creation of the Macintosh to Vint Cerf’s role at Google as Chief Internet Evangelist, which often includes memorializing and storytelling.

I want all of these groups to be the audience for this work, especially that latter two who are not especially well-known as patrons of the arts and who have a particular interest in the meaning and interpretation of this story. In my last post, I described the “complex terrain of political and aesthetic geography” provided by princely patrons as being creatively stimulating for artists as they tried to work “within and against it”. Similarly, I want these people as patrons because they have powerful opinions about the meaning of this story in itself. The friction and feedback they provide as I attempt to render it into meaning and myth will make for better art and a more lasting monument to this story of invention.