Pathways: Cutting-Edge Virtual Reality Technology Impacts Surgery Planning, Patient Care
UICOMP faculty are using MRI and CT data to generate an exact replica of anatomy. This can be used to create a three-dimensional model of the anatomy that physicians can use to better understand individual cases. It is especially helpful in surgery planning and allows surgeons to see “up close and personal,” albeit virtual. Peoria is a worldwide leader in using this technology. With the use of this cutting-edge VR technology, physicians can examine and manipulate the images to get a precise 3D view that current MRI and CT data in and of itself cannot provide.
The concept/technology was developed in the laboratory at Jump Simulation (a collaboration between OSF HealthCare and UICOMP), but is now being applied to complex cases which gives physicians and surgeons a remarkable advantage in planning and ultimately provides better care for patients.
For years, physicians have used tools like x-ray, ultrasound, MRI and CT in an attempt to visualize inside the human anatomy. The use of this technology allows physicians to see what is going on.
Leading the creation and application of this technology in Peoria is Matthew Bramlet, MD, assistant professor of clinical pediatrics, who explains the evolution of this capability began with the introduction of consumer-based 3D printing in 2014. The wide availability of 3D printing provided easier access to a technology that demonstrated the benefit of interacting with complex anatomy derived from MRI and CT images.
“The work of generating 3D models involves translating what the radiologists see when they look at each 2D image and assigning a value to the image. As each image, with segmented values assigned to the different tissues, we can then stack those together to create a 3D model of the patient-specific anatomy. This process objectifies in digital form, the 3D model as seen in the “minds-eye” of the radiologist,” Bramlet explains.
“The real value came when we took the existing CT and MRI data sets that had the 3D information hiding in it, and in our lab, we generate a digital twin of the patient,” Bramlet says. “This allows the surgeon, initially in 3D print form, but now in a more advanced VR form, to assess that internal anatomy much as they will see it in the OR. What we’re achieving is an improved mental image in the surgeon’s mind, especially on these complex cases, so that the surgeon can encounter the unexpected before they go to the OR.”
Surgeon Sonia Orcutt, MD, assistant professor of clinical surgery, echoes the advantages of VR. “I think what VR helps with is giving us, as surgeons, the ability to see the anatomy before we're actually in the OR,” Orcutt says. With two-dimensional images like CT scans or MRIs which are the current standard, we have to build a model in 3D in our mind, which is good but not perfect. With VR sometimes we can see relationships with other structures better than if we were imagining it, which can help plan surgeries better.”
Not only does this help avoid surprises, but it can also assist with overall planning and approach to surgery. Application of the technology started with congenital heart patients but has evolved to include cancer care, both pediatric and adult. Bramlet says nearly 50 percent of complex cases that use this technology for surgery planning find the surgeon changing something about their approach.
Orcutt shares that VR has influenced her surgery plans. ”I have in the past changed my incision for surgeries or changed the approach itself, in terms of how much I've removed in the OR,” she says. Even if a surgeon doesn’t necessarily change her plan, knowing what to expect offers clear advantages. “Having already 'seen' the anatomy before helps to know how and where to watch out for critical structures,” she adds.
“In terms of ultimate impact on patient care, overall I think this is safer for us as surgeons and helps us be better prepared. It may also reduce our operative times, which will help patients,” Orcutt says. “I'm very glad we have access to this amazing technology here in Peoria.”
Marc Knepp, MD, assistant professor of clinical pediatrics and medicine (UICOMP Class of 2004), has applied the technology firsthand with his patients with congenital heart defects, heart problems present at birth. “In congenital heart, there’s a myriad of lesions and every child is a different size, shape, form, and so trying to do a standard approach for every child on the operation or procedure can be challenging,” Knepp explains. “When we’ve taken these cases and put them in a virtual reality or a 3D model, we can actually make a difference and individualize the care so the procedure goes faster, it’s smoother, and complications are already predicted even before they happen.”
The 3D modeling and VR technology also provide a great tool for teaching and educating patients and families as well as students and residents. “They get the opportunity to learn from three- dimensional modeling that no one else has or very few centers have,” he says. “And that has been eye-opening for me. I remember my first patient who got to see her heart in virtual reality, walk through her heart. There were tears running down her face, and she said, ‘I’ve never understood my heart like this.’ And she had been our patient for years and years, but that virtual reality made such a difference.”
“This has been quite revolutionizing for our patients. They have access to something that is on the cutting edge here in Peoria,” Knepp says. Bramlet echoes the rarity of the technology.
“Currently, this technology is not a standard capability in medicine even today. Between UICOMP and OSF HealthCare, we’ve really combined to bring incredible cancer care to the forefront here in this area that patients can’t go elsewhere and find.”
This article is part of the Summer 2021 edition of Pathways. Read the full issue.