Medical Holograms Are Now Part of the Surgeon’s Toolkit

Holograms aren't just for Princess Leia anymore. Interactive technology hitting the market now can help doctors examine vital organs using 3D displays that hover over a desktop screen.

This week, a company called EchoPixel announced that the U.S. Food and Drug Administration has cleared its True3D Viewer for use in diagnostics and surgical planning. The software platform converts existing 2D medical imaging data such as MRI and CT scans into fully interactive virtual reality images. With the system, doctors can view, manipulate and dissect body parts that are re-created in mid-air above an ordinary desktop. 

The use of 2D digital scans revolutionized medicine, because they allowed doctors to see an individual patient's anatomy without cutting into the body. "But when a doctor evaluates them, they are looking at a series of 2D slices and trying to create that 3D anatomy in their mind,” says Sergio Aguirre, founder and CTO of EchoPixel. “Doctors are focusing energy on solving this 3D problem instead of the clinical problem at hand, and we think this software will help them get a clearer view of the problem more quickly.” 

Other systems, like GE's Vivid E9 with XDclear, already compile such images to produce 3D visuals that appear much like the real thing, and they even have 3D properties that allow them to be rotated or taken apart. But they are still limited to display on a flat screen. EchoPixel appears to take 3D imaging a step further by generating interactive holograms.

Experts have become very good at reading 2D images and manipulating 3D representations on a flat screen, so holograms may not add a huge advantage in some applications, says Sandy Napel, co-director of Stanford University's Radiology 3D and Quantitative Imaging Laboratory. But there are specific procedures that EchoPixel may be primed to improve. For instance, the technology is already being tested at the University of California, San Francisco, for virtual colonoscopies—an alternative to the unpopular procedure in which a colonoscope is inserted and manipulated within the human body.

“You want to simulate what a doctor would see while examining the interior surfaces of the colon with a colonoscope, and you want to see 100 percent of the interior surface of this long, curving tube that is the large intestine,” Napel explains. “Using CT scan images, this technology can really reproduce this tube-like colon, re-create it floating in space, and nothing has to actually enter the body. You can rotate the image at different angles, cut it in half and search the interior surface for polyps. It's a way of visualizing the colon that has great potential to improve how quickly you could look at 100 percent of the interior.”

True 3D medical imaging may also benefit doctors who need to visualize abnormal or complex 3D structures, like the mess of broken and displaced bones that can result from trauma due to a motor vehicle accident. “A surgeon that's going to plan to remove fragments and fix that kind of injury might benefit by viewing a true 3D representation of what they will actually see when they have the patient in the operating room,” Napel says. “I think having, say, a 3D pelvis floating above a desktop, where you could see all the actual fractures and displacements, could have great potential for surgical planning.”

Heart conditions in very young patients are another area where medical holograms could shine. “The heart is a complicated structure, but every medical student can draw a picture of a normal heart,” Napel says. “However, when you have narrowings, aneurisms, congenital abnormalities—being able to visualize these in 3D could be really helpful. Think about kids born with genetic defects that cause the heart to develop abnormally. A surgeon is going to go in there and operate on a very young human being and hopefully make a correction. They'll get a careful report from a radiologist, saying that some blood vessels are connected here and they should be there, and the surgeons can see it also on CT scans, but not the same way they are going to see it in the operating room." Having a 3D preview of what they'll see when they start surgery could help the doctor understand the situation much more quickly.

And while years of education and experience enable doctors to work effectively with 2D medical images, the rest of us often have a very hard time deciphering them. That highlights another intriguing application for virtual reality—patient education. It's a promising niche in an era when the public is demanding more information from their medical providers.

“Patients want to know exactly what a doctor is going to do to them. I think it could be pretty neat and pretty powerful for patients going in for complex surgeries to see exactly what it is the surgeon is going to see and what they are going to do during the operation.”

Brian Handwerk | READ MORE

Brian Handwerk is a science correspondent based in Amherst, New Hampshire.