Reconstructing visual experiences from brain activity

Despite its length and hard-to-pin-down clunkiness, I always liked Until the End of the World, Wim Wenders’ rambling near-term sci-fi film about (among other things) the psychological impact of a technology that enables the recording and playing back of one’s dreams. In the film, the characters become addicted to the technology, recording their dreams every night and spending more and more of their waking hours reviewing the recordings until their lives are consumed by reflection.

Other movies, like Kathryn Bigelow’s Strange Days, explore the unsettling social implications of a technology that enables people to digitally capture and play back what their brain sees and hears and feels. More recently, Charlie Brooker’s brilliant Black Mirror (episode 3, “The Entire History of You“) presents a plausible vision of how such a technology could become ubiquitous, and the devastating effects it will have on privacy, intimate relationships, and the way we remember our lives.

And so it was somewhat disconcerting to come across this research conducted at Berkeley’s Gallant Lab. The researchers describe the project as follows:

As you move through the world or you watch a movie, a dynamic, ever-changing pattern of activity is evoked in the brain. The goal of movie reconstruction is to use the evoked activity to recreate the movie you observed. To do this, we create encoding models that describe how movies are transformed into brain activity, and then we use those models to decode brain activity and reconstruct the stimulus. (UCB Gallant Lab)

Right now, to capture these recordings, the subject needs to be positioned inside a giant fMRI machine, so we’re a little way off from the dystopias described by the narratives of Strange Days and Black Mirror. But as the researchers at the Gallant Lab write, “both the technology for measuring brain activity and the computational models are improving continuously. It is possible that decoding brain activity could have serious ethical and privacy implications downstream in, say, the 30-year time frame.”

Strange days, indeed.

Berkeley Talk: Transforming Community Through Pervasive Play

I will be speaking at the Berkeley Center for New Media on February 2nd, 2012, at 5PM in the BCNM Commons (340 Moffitt). Here’s the description of the talk:

In this talk, Jeff Watson will present Reality Ends Here (2011), a pervasive alternate reality game designed to effect immediate change in the community of learners at the USC School of Cinematic Arts (SCA). Over the course of the project’s 120 day run, collectible cards, rumors, secret websites, and a mysterious black flag drew more than 150 students into an intense underground social game involving collaboration, strategy, and artistic experimentation. By connecting students to one another in unpredictable and serendipitous ways, and by providing a framework for meaningful play and performance, the game transformed a collection of heavily siloed academic divisions into a productively chaotic and interdisciplinary community of practice. Drawing on the research and methodology underlying the design, implementation, and assessment of Reality Ends Here, Watson will argue for the transformative potential of pervasive game interventions across a range of domains, from education and public policy, to activism, innovation, and beyond (Berkeley Center for New Media).

UPDATE: Slides posted here.