Videos reconstructed from brain scan

Using MRI brain scans, computational models and a large quantity of YouTube videos, University of California, Berkeley researchers have demonstrated how people’s dynamic visual experiences can be reconstructed. The image at right shows the source video image and the reconstructed image. The videos themselves can be seen on YouTube.

Reporting on their work in the journal Current Biology, the researchers say it is a major step toward reconstructing internal imagery. “We are opening a window into the movies in our minds,” said neuroscientist and study co-author Jack Gallant. Importantly, the technology can only reconstruct movie clips people have already viewed. However, Gallant believes the work paves the way for reproducing the movies inside our heads that no one else sees, such as dreams and memories.

Previously, Gallant and fellow researchers recorded brain activity in the visual cortex while a subject viewed black-and-white photographs. They then built a computational model that enabled them to predict with accuracy which picture the subject was looking at. In their latest experiment, the researchers say they have solved a much more difficult problem by actually decoding brain signals generated by moving pictures.

Co-researcher Shinji Nishimoto said that decoding video from the brain was a time consuming business, with volunteers having to remain still inside the MRI scanner for hours at a time. While in the scanner, they watched two separate sets of Hollywood movie trailers.

The MRI scanner was used to measure blood flow through the visual cortex. Computationally, the cortex was considered to be made of small cubes known as volumetric pixels, or “voxels.” “We built a model for each voxel that describes how shape and motion information in the movie is mapped into brain activity,” Nishimoto explained.

The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity.

Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding some 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject. Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a reconstruction of the original movie.

Ultimately, Nishimoto said, scientists need to understand how the brain processes dynamic visual events that we experience in everyday life. “We need to know how the brain works in naturalistic conditions,” he said. “For that, we need to first understand how the brain works while we are watching movies.”

Related:
MRI scans reveal diverse female sexual sensory palette
MRI scans predict pop music success
Pain detector accuracy surprises medicos
This is your brain on androids

Source: University of California, Berkeley

, , ,

Comments are closed.

MRI Up Productivity Pin Button Vintage picture

MRI Up Productivity Pin Button Vintage

$19.99



Vintage Bic Pen Advertisement Film MRI Center Llc Blue White Plastic picture

Vintage Bic Pen Advertisement Film MRI Center Llc Blue White Plastic

$19.49



CPA ak morocco fez view of town, taken from Beni mri (10263) picture

CPA ak morocco fez view of town, taken from Beni mri (10263)

$5.87



Gold MRI Excellence Award Green Marble & Dark Hardwood Wall Trophy Black Frame picture

Gold MRI Excellence Award Green Marble & Dark Hardwood Wall Trophy Black Frame

$45.00



1994 Press Photo MRI units assembled at Intermagetics General Corp, Latham, NY picture

1994 Press Photo MRI units assembled at Intermagetics General Corp, Latham, NY

$10.00



Real MRI M. R. I. Film Physical Scan Image 14

Real MRI M. R. I. Film Physical Scan Image 14" X 17" Medical X-ray

$28.99



RAILROADER'S HANDBOOK (C/MRI Applications) Volume 2 Signaling Systems 2010 picture

RAILROADER'S HANDBOOK (C/MRI Applications) Volume 2 Signaling Systems 2010

$35.00



Funny Mug MRI Mad Scientist Creepy 12 oz picture

Funny Mug MRI Mad Scientist Creepy 12 oz

$9.99



Medical, Science, Research License Plate, MRI neuroimaging, scanning picture

Medical, Science, Research License Plate, MRI neuroimaging, scanning

$49.99



London 2012 Olympic Worldwide Partner General Electric MRI Scanner Enamel Pin picture

London 2012 Olympic Worldwide Partner General Electric MRI Scanner Enamel Pin

$11.99



Powered by WordPress. Designed by WooThemes