By Bill Murphy/Science Correspondent © 2013
UC Berkeley neuroscientists addressed that question in a paper published September 22, 2011 in the online journal Current Biology. The team consisting of Shinji Nishimoto, An T.Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu and Jack L. Gallant at the university developed a system to capture activity in the visual cortex of the human brain and reconstruct it as digital video clips that can be played back.
While questions remain regarding what humans perceive visually versus what is imagined in our minds, it is theorized that the process used to reconstruct images we see could be used to record dreams. Professor Jack Gallant – UC Berkeley neuroscientist and coauthor of the research – suggest that movies we watch may in fact result in brain signals similar to what we experience when dreaming.
In the Volume 1, Number 3 December 2012 issue of Lucid Dreaming Experience magazine, I presented other research by neuroscientists using a functional Magnetic Resonance Imaging (fMRI) system for lucid dream research and included additional details of how they operate that can be referenced by those interested in learning more about how these systems function.
Detecting metabolic and vascular responses within the brain is useful, but these two processes are very slow compared to the fast firing of the hundreds of thousands of neurons that occurs during all mental activity. This limitation has required most researchers to limit the process of decoding brain signals within the visual cortex to what happens when viewing a static image.
However the UC Berkeley team wanted to reconstruct the visual experiences from what they called ‘natural movies’, which is what occurs when watching life unfold before your eyes, or while viewing recorded video clips, and possibly what happens during visualized dream sequences.
The self imposed challenge they accepted was to understand what happens while engaged in experiencing natural, dynamic visual events so the volunteers (which included the researchers themselves) watched a large random library of video clips to track and record how the brain responds to various types of moving images. According to Nishimoto, this was addressed by developing a two-stage model that separately describes the underlying neural population and blood flow signals.
A database they referred to as a dictionary was assembled that associated brain activity in response to viewing the various shapes, edges, and motion of numerous videos. This was used to create a computer model that would predict how the brain would respond to a wide variety of visual stimulation.
They now had an encoding system that could be used to reconstruct moving images based on the dictionary of recorded brain activity. By analyzing the brain activity of the subjects while watching short clips and movie trailers, the computer models would assemble new digital video files by averaging and assembling images obtained from videos posted on the internet.
Although the reconstructed images are not exact matches for what a person would be seeing or imagining, the result is a method that works to reassemble an approximate version of the original mental image based on brain activity responding to visual stimuli. The limitation imposed by the slow process of using an fMRI has been answered with some degree of success despite the reconstructed video appearing blurry due to the recreated video being assembled from hundreds of samples of videos that only share broad similarities of movement and shapes.
Following is an example of a single frame of a video viewed on the left with the reconstructed single frame generated by the computer on the right: Neuroscientists are in general agreement that all mental processes have a consistent neurobiological basis. If correct, then it stands to reason that recording dreams may someday utilize a reliable method that has its origins in the exciting work we see being conducted today.
The research is tedious, has limitations, and there is some debate in the lucid dreaming community regarding the methods used described here, but the work by the team at UC Berkeley is promising. For those involved in lucid dream research, it is especially encouraging. Advancements in medical technology continues and an improvement to what is currently available would be welcome as fMRI systems are expensive, noisy, require subjects to remain still and are not designed to detect the electrical impulses in the brain.
But with determination, the UC Berkeley team has made an important contribution to recording mental visualization and reconstruction that can be viewed by others. As a researcher of lucid dreams, I look forward to what the future may reveal for those who strive to document dreams with content created intentionally.
For an inspiring 30 second video of the work by the researchers mentioned in this article, view the following clip posted on YouTube: https://www.youtube.com/watch?v=OKpO_v76kVs
More information on the work of the Gallant Lab at UC Berkeley can be found at the following website: https://sites.google.com/site/gallantlabucb/publications/ nishimoto-et-al-2011
