Brainlab and Magic Leap: Providing More Insights to Advance Surgery


4 min read

The idea of incorporating spatial computing systems into medicine seemed like a futuristic dream just a decade ago. More than 120 Magic Leap 1 headsets with Brainlab's Mixed Reality Viewer are in clinical use around the globe today. There are a variety of uses for these highly advanced visualization tools, helping medical professionals with anything from surgical planning and medical training to offering patients an easier way to see and understand their diagnosis and treatment.



On Magic Leap 1, the Mixed Reality Viewer transforms any work area into a virtual, interactive space that reinforces 3D visualization, interaction and understanding. In the Magic Leap 1, built-in sensors map the room, so that clinicians can realistically imitate the planned surgery, placing 3D anatomical models and 2D image slices anywhere in the room, whether it's an office, a meeting room or even at home. The scheme enables up to four individuals, either in the same room or even remotely within the same network, to collaborate and interact with each other in real time.


"We have found that when using the Mixed Reality Viewer software on Magic Leap 1 during scheduling, we can effectively reduce the risk of surgical complications in almost any case," explained Veit Braun, MD, director and professor at Jung-Stilling Hospital, part of the Diakonie Hospital network in Siegen, Germany. Before using this technology, we see less of the distinct complications we saw. It also helps to shorten surgical periods under anesthesia and patient time, which is a great benefit.


One area where spatial computing is becoming a staple for viewing and interacting with patient imagery and data is surgical planning. In order to support treatment planning, clinicians identify significant value in surgical precision and time savings from using the Mixed Reality Viewer on Magic Leap 1. For multiple indications of treatment, including aneurysms and skull base tumors, the technology is already in daily clinical use in Europe. Given the complexity and need for detailed planning, surgeons are increasingly integrating Brainlab's Mixed Reality Viewer on Magic Leap 1 into their clinical routine for cervical fractures and craniomaxillofacial surgery. The ability to enlarge the anatomy and virtually walk around, or even within it, provides a completely new perspective that is so close to reality and so vivid that over the next few years, the industry will quickly realize a paradigm shift.



How AR Surgical Planning Works Actually ?

The surgeon begins by creating a plan using surgical planning software to allow the viewing of a case in spatial 3D. From available radiology imaging, like MRI and CT, a 3D model of the patient's anatomy is generated. In the image, the software then identifies and outlines the different anatomical structures and the surgical target, such as a tumor or an arteriovenous malformation (AVM).


The surgeon can begin to plan the optimal approach to the surgical target with these structures identified in the 3D model, avoiding any critical anatomy. Once this plan is created, the surgeon can then mount a head-mounted augmented reality display, such as the Magic Leap 1, and move it from the 2D screen of the workstation and place it in the room. They can walk around it, evaluate the plan from all angles, and even invite other clinicians to participate together in real-time to review and discuss the plan.


Although this technology is considered to be the pinnacle of today's surgical plan review, the concept of evaluating the surgical situation dates back almost more than fifty years. In orthopedics, the first hints of using a spatial viewing technology in surgery began when Robert Mann suggested using VR in 1965 to allow surgeons to test multiple approaches for orthopedic cases. I This technology is more real than ever as it expands into a wide range of surgical subspecialties quickly until 2020.


In neurosurgery, Spatial Computing

Neurosurgery is extremely complex because of the close proximity of critical structures in the brain. The surgical outcome and even the quality of life of the patient can be drastically affected by one small deviation from a surgical plan. Technology, like augmented reality, that allows the surgeon to review a detailed spatial 3D plan before surgery is a crucial advance in ensuring that the treatment is performed exactly as intended.


I use the Brainlab Mixed Reality Viewer on Magic Leap 1 during the planning process for every aneurysm case. If I need to do clipping or coiling, it helps me easily determine. For clipping, viewing in mixed reality is particularly helpful. The technology shows you precisely where the neck of the aneurysm and aneurysm are located. When planning the approach, it is not only extremely helpful, it is also fascinating to virtually see and interact with the anatomy. Veit Braun, MD, Head of Neurosurgery at Jung-Stilling Diakonie Klinikum, Siegen, Germany


Another clear benefit of planning with an Augmented Reality tool, like the Magic Leap 1 software from Brainlab, is that the surgeon knows exactly what they are going to do from the first cut. It is possible to complete the operation more quickly and with a reduced risk of complications. Since prolonged time under anesthesia increases the risk of complications during and after the procedure, surgeries that are more effective benefit the patient immensely.


Interdisciplinary planning with Augmented Reality for CMF cases

In craniomaxillofacial (CMF) surgery, the incorporation into the planning process of the use of Augmented Reality technologies supports clinicians in their decision-making. Multiple clinicians can attend a session with Magic Leap 1 and observe the same patient images at the same time, even remotely. This ensures that the software actually promotes interdisciplinary planning.


I believe the next big step in virtual surgical planning, patient education, as well as medical education will be augmented reality applications. In combination with planning tools, augmented reality enables craniomaxillofacial surgeons to segment out and visualize complex anatomy before stepping into the operating room in a three-dimensional environment. The fact that, in the same environment, multiple users can interact with these virtual objects is a game changer. 


For example, during a tumor board meeting, surgeons, radiologists, and radiation oncologists may discuss resection with adjuvant radiotherapy to identify the best approach. The surgery can progress quickly as they know when to stop and, in turn, move the patient to radiation therapy because the surgeon knows and has seen the extent of the resection in advance.


Stereotactic Functional Surgery

In many phases of clinical workflow, functional neurosurgery applications, such as deep brain stimulation (DBS) and stereo-electroencephalography (SEEG), require interdisciplinary coordination. From preoperative planning all the way to postoperative patient management, neurosurgeons, epileptologists and neurologists need to work together.


Functional neurosurgeons and epileptologists are supported by Augmented Reality technology to combine their knowledge to create an optimal trajectory plan prior to a SEEG case. These types of cases consist of the implantation of up to 20 electrodes in patients with drug-refractory epilepsy to identify epileptic focal points. Collaboration between these specialists can contribute to enhanced results.


Many cases of SEEG are three-dimensional complex implantations planned by an interdisciplinary team. Because the implantations are customized according to clinical, imaging and EEG results, each system is different. Especially for this use-case the visualization in mixed reality turns out to be very helpful for the team discussion during planning.” – Peter Reinacher, MD, Consultant, Department of Stereotactic and Functional Neurosurgery, Medical Center – University of Freiburg, Germany, and Group Leader (ATTRACT), Fraunhofer Institute for Laser Technology, Aachen, Germany.


Hours or days after deep brain stimulation surgery, neurologists program the electrodes to ensure that they stimulate the brain of the patient as planned. On a new level, this technology can contribute uniquely to their understanding of the patient's anatomy, supporting visualization-based programming instead of traditional approaches to trial and error.


Vascular operation

In a spatial computing headset such as Magic Leap 1, the 3D reconstructions that the clinician can view help greatly in their understanding of very complex anatomy. A deep-seated kidney artery aneurysm, in which a true-to-life representation provides a much clearer image of the case than two-dimensional scans alone, would be an example of vascular surgery.


The surgeon will have access to a range of data in the future when spatial computing and related technologies come into regular intraoperative use, without ever having to look away from the surgical situation. It is possible to display bloodwork outcomes, stent graft measurements, EKG, preoperative images, or even radiation exposure to the surgical team, turning the field of view of the surgeon into a "virtual cockpit" v.