Reconstructing visual experiences from brain activity

Despite its length and hard-to-pin-down clunkiness, I always liked Until the End of the World, Wim Wenders’ rambling near-term sci-fi film about (among other things) the psychological impact of a technology that enables the recording and playing back of one’s dreams. In the film, the characters become addicted to the technology, recording their dreams every night and spending more and more of their waking hours reviewing the recordings until their lives are consumed by reflection.

Other movies, like Kathryn Bigelow’s Strange Days, explore the unsettling social implications of a technology that enables people to digitally capture and play back what their brain sees and hears and feels. More recently, Charlie Brooker’s brilliant Black Mirror (episode 3, “The Entire History of You“) presents a plausible vision of how such a technology could become ubiquitous, and the devastating effects it will have on privacy, intimate relationships, and the way we remember our lives.

And so it was somewhat disconcerting to come across this research conducted at Berkeley’s Gallant Lab. The researchers describe the project as follows:

As you move through the world or you watch a movie, a dynamic, ever-changing pattern of activity is evoked in the brain. The goal of movie reconstruction is to use the evoked activity to recreate the movie you observed. To do this, we create encoding models that describe how movies are transformed into brain activity, and then we use those models to decode brain activity and reconstruct the stimulus. (UCB Gallant Lab)

Right now, to capture these recordings, the subject needs to be positioned inside a giant fMRI machine, so we’re a little way off from the dystopias described by the narratives of Strange Days and Black Mirror. But as the researchers at the Gallant Lab write, “both the technology for measuring brain activity and the computational models are improving continuously. It is possible that decoding brain activity could have serious ethical and privacy implications downstream in, say, the 30-year time frame.”

Strange days, indeed.

Posted in Blog | Tagged , , , , , | Comments Off on Reconstructing visual experiences from brain activity

Berkeley Talk: Transforming Community Through Pervasive Play

I will be speaking at the Berkeley Center for New Media on February 2nd, 2012, at 5PM in the BCNM Commons (340 Moffitt). Here’s the description of the talk:

In this talk, Jeff Watson will present Reality Ends Here (2011), a pervasive alternate reality game designed to effect immediate change in the community of learners at the USC School of Cinematic Arts (SCA). Over the course of the project’s 120 day run, collectible cards, rumors, secret websites, and a mysterious black flag drew more than 150 students into an intense underground social game involving collaboration, strategy, and artistic experimentation. By connecting students to one another in unpredictable and serendipitous ways, and by providing a framework for meaningful play and performance, the game transformed a collection of heavily siloed academic divisions into a productively chaotic and interdisciplinary community of practice. Drawing on the research and methodology underlying the design, implementation, and assessment of Reality Ends Here, Watson will argue for the transformative potential of pervasive game interventions across a range of domains, from education and public policy, to activism, innovation, and beyond (Berkeley Center for New Media).

UPDATE: Slides posted here.

Posted in Blog | Tagged , , , , , | 4 Comments

Play, writing, and the pleasures of complex dynamic systems

Writer and game designer Andrea Phillips, who I interviewed in this space a few years back, recently wrote a blog post about the evolution of her writing process, describing “the way that my creation of stories and my creation of games have come to use the same general process.” The gist of the post is something like this: developing stories amounts to something very similar to developing games in terms of the way that both forms demand striking a kind of systemic balance. An unbalanced game will be exploited by its players, or, as in the example Phillips uses of a game which over-incentivizes certain play actions through its point system, will bring about undesired behaviors that detract from the core experience. Similarly, narrative figures fail to generate their intended effects unless they are finely “balanced” toward specific ends. This could be illustrated by the canonical example of how showing a ticking time bomb hidden beneath a table at the beginning of a sequence will generate suspense, but if it is shown only right before it explodes, the result will be mere shock. In both cases — games and narratives — simple changes in sequence, tone, and fact can have enormous impact on the system as a whole.

Some of my own first inklings of this sort of systems thinking came about when I was learning how to write JavaScript. One of the first projects I did was a kind of “random log line generator” that put together snippets of beginnings, middles, and ends to create surprising (and often absurd) pseudo-random stories. As I worked to make the program do more sophisticated things — things like check if there had been a car mentioned in an earlier part of the story, and if there had, bring it back in later in the story — I began to see more clearly how traditional fixed linear stories (at the time I was working on various screenplay projects) were in fact complex dynamic systems (at least in terms of the development process — though of course as far as their relationship to spectators goes, they remain so long after they are “finished”). Making a change in one part of the text has cascading effects throughout the whole, changing meanings, altering stakes, and opening (or closing) lines of possibility. It seems obvious now, but for me it also felt like a breakthrough.

Maybe that discovery was part of why I became interested in participatory and environmental media broadly and game design more specifically. The thrill of watching those possibilities open and close and those changes ripple through the system was something I wanted to design for. Why should authors have all the fun playing with the pieces and seeing how things shake out differently as the constituent elements of a story environment are changed? As Phillips puts it, it’s a wonderful game to imagine “how else we might have assembled the same cogs and gears to make [the clockwork machine of a story] run faster or quieter or keep time better.”

This pleasure, I think, is at the heart of game play, not just game design. It’s a unique kind of pleasure that comes from a feeling of real agency, of having one’s actions effect tangible consequences upon a system, and of discovering the new and unforeseen challenges associated with those consequences — and it’s what keeps me passionate about writing, designing, and playing alike.

Posted in Blog | Tagged , , , , | 3 Comments

Metropolis II

Chris Burden’s Metropolis II opens this week at LACMA. According to Burden, “The noise, the continuous flow of the trains, and the speeding toy cars, produces in the viewer the stress of living in a dynamic, active and bustling 21st Century city.” And also, I would add, the thrill and the wonder…

Here are a few stats on the piece:

  • The cars are attached by a small magnet to the conveyor belt that brings them to the crest.
  • The only motorization of the cars is the conveyor belt to the top.
  • Once the cars cross over the crest and head downward, their entire movement is by gravity.
  • They travel at a scale speed of 240 mph, plus or minus.
  • The tracks they take are Teflon coated to reduce friction.
  • The tracks are beveled at 7 degrees to give added torque for speed when
    they come through corners and curves.
  • The trains are out of the box electric train sets that run on electricity.

(Unframed)

See also: Big Wrench.

Posted in Blog | Tagged , , , , , | 1 Comment

“In its efforts to be “safe rather than sorry,” precaution becomes myopic. It tends to maximize only…”

“In its efforts to be “safe rather than sorry,” precaution becomes myopic. It tends to maximize only one value: safety. Safety trumps innovation. The safest thing to do is to perfect what works and never try anything that could fail, because failure is inherently unsafe. An innovative medical procedure will not be as safe as the proven standard. Innovation is not prudent. Yet because precaution privileges only safety, it not only diminishes other values but also actually reduces safety.”

Kevin Kelly – What Technology Wants

Posted in Quotes | Comments Off on “In its efforts to be “safe rather than sorry,” precaution becomes myopic. It tends to maximize only…”

“Ubiquitous systems lend themselves easily to—indeed, redefine—surveillance. However discrete they…”

“Ubiquitous systems lend themselves easily to—indeed, redefine—surveillance. However discrete they may be at their design and inception, their interface with each other implies a domain of action that extends from the very contours of the human body outward to whatever arbitrarily large civic space can be equipped with the necessary sensors and effectors. In short, there is no current technology with greater potential to support authoritarian and totalitarian social engineering, and the limitation otherwise of choice.”

Adam Greenfield – All watched over by machines of loving grace

Posted in Quotes | Comments Off on “Ubiquitous systems lend themselves easily to—indeed, redefine—surveillance. However discrete they…”

An essential archipelago of opportunity

Nicholas de Monchaux’s WPA 2.0 entry, Local Code : Real Estates uses geospatial data to map the thousands of abandoned city-owned lots scattered across North American cities. But this is more than just a data viz project: de Monchaux conceives of these spaces as “an essential archipelago of opportunity” for making cities more livable, functional, and sustainable. The project proposes a provocative union of urban environmental sensor data, citizen participation (presumably captured via social media), and parametric design software:

Using parametric design, a landscape proposal for each site is tailored to local conditions, optimizing thermal and hydrological performance to enhance the whole city’s ecology—and relieving burdens on existing infrastructure. Local Code’s quantifiable effects on energy usage and stormwater remediation eradicate the need for more expensive, yet invisible, sewer and electrical upgrades. In addition, the project uses citizen participation to conceive a new, more public infrastructure as well —a robust network of urban greenways with tangible benefits to the health and safety of every citizen. (Nicholas de Monchaux)

Related: WPA 2.0 Exhibition

Posted in Blog | Tagged , , , | Comments Off on An essential archipelago of opportunity

Mogees: Realtime Gesture Recognition with Contact Microphones

Bruno Zamborlin’s Mogees project — “an interactive gestural-based surface for realtime audio mosaicing” — implies all sorts of interesting futures for musical instruments and performance:

http://youtu.be/erz-9f4M9B4

Via @elejansen

Posted in Blog | Tagged , , , , , | 1 Comment

Untitled

Untitled

Filter: Hudson

Posted in Photos | Tagged | Comments Off on Untitled

“Thomas Maldonado … saw the design process as a system embodying both scientific-based and…”

“Thomas Maldonado … saw the design process as a system embodying both scientific-based and intuitive-based thinking. He considered that while design is indeed an art, the designer is not solely an artist. Aesthetic considerations were no longer the primary conceptual basis of design. The professional designer would be an “integrator” with responsibility for integrating a large number of specialties in addition to aesthetics, mostly the diverse requirements of materials, manufacturing and context of product use, as well as considerations of usability, identity and marketing.”

Wikipedia: Ulm School of Design

Posted in Quotes | Comments Off on “Thomas Maldonado … saw the design process as a system embodying both scientific-based and…”