Reconstructing visual experiences from brain activity

Despite its length and hard-to-pin-down clunkiness, I always liked Until the End of the World, Wim Wenders’ rambling near-term sci-fi film about (among other things) the psychological impact of a technology that enables the recording and playing back of one’s dreams. In the film, the characters become addicted to the technology, recording their dreams every night and spending more and more of their waking hours reviewing the recordings until their lives are consumed by reflection.

Other movies, like Kathryn Bigelow’s Strange Days, explore the unsettling social implications of a technology that enables people to digitally capture and play back what their brain sees and hears and feels. More recently, Charlie Brooker’s brilliant Black Mirror (episode 3, “The Entire History of You“) presents a plausible vision of how such a technology could become ubiquitous, and the devastating effects it will have on privacy, intimate relationships, and the way we remember our lives.

And so it was somewhat disconcerting to come across this research conducted at Berkeley’s Gallant Lab. The researchers describe the project as follows:

As you move through the world or you watch a movie, a dynamic, ever-changing pattern of activity is evoked in the brain. The goal of movie reconstruction is to use the evoked activity to recreate the movie you observed. To do this, we create encoding models that describe how movies are transformed into brain activity, and then we use those models to decode brain activity and reconstruct the stimulus. (UCB Gallant Lab)

Right now, to capture these recordings, the subject needs to be positioned inside a giant fMRI machine, so we’re a little way off from the dystopias described by the narratives of Strange Days and Black Mirror. But as the researchers at the Gallant Lab write, “both the technology for measuring brain activity and the computational models are improving continuously. It is possible that decoding brain activity could have serious ethical and privacy implications downstream in, say, the 30-year time frame.”

Strange days, indeed.