Japan Succeeds in Image Reconstruction from Brain Signal Data
Tokyo, Nov. 30 (Jiji Press)--A Japanese team has developed a technology to reconstruct mental images of scenery and objects from brain signal data acquired through functional magnetic resonance imaging, or fMRI, equipment.
Previously, reconstruction of such images was achieved only when subjects were seeing objects directly with the eyes or imagining letters and geometric patterns.
The team, including the National Institutes for Quantum Science and Technology, or QST, said it has become the first to successfully reconstruct arbitrary mental images from brain signal data.
The technology is expected to facilitate the development of new communication devices while supporting research to elucidate the mechanism of how humans see dreams and hallucinations.
The team's study has been published in the electronic edition of the international journal Neural Networks.
Kei Majima, a QST researcher, and other team members asked three participants in the team's experiment to see 1,200 natural images and measured their brain signals using fMRI.
In the experiment, brain signals caught while the participants were viewing the images were converted into digital data one by one.
The team then used artificial intelligence technology to create tables of numerical values for the 1,200 images that represent the characteristics of the images.
Based on the data and the tables, the team built a translation program that creates numerical value tables from data of brain signals obtained when participants are viewing arbitrary images.
The three participants were shown 25 images different from the 1,200 images used to acquire data.
They were then asked to bring one of the 25 images to their minds each time while the images were hidden from them.
In the meantime, their brain signals were observed with fMRI equipment.
Using the translation program, the team created from the brain signal data numerical value tables of mental images that the participants had in their minds.
The team then ran an image generation AI engine to produce new images.
The engine repeated work to correct the images about 500 times until the numerical value tables of the newly drawn images come to resemble those of the mental images, so that the engine can generate "restored" images.
Evaluated similarities between the restored and original images significantly improved from the levels achieved with the conventional method that does not use image generation AI technology.
When the team had participants imagine a leopard's face, images restored by the engine captured characteristics of the animal, such as the shape of ears, leopard pattern and part of the face around the mouth.
"This is the first achievement of humankind to peek into other people's heads," Majima said.
"We hope this will deepen understanding of what's inside the mind."
But he also said it is necessary to consider ethical problems that may be posed by the technology.
END
