eISSN: 2081-2841
ISSN: 1689-832X
Journal of Contemporary Brachytherapy
Current Issue Archive Supplements Articles in Press Journal Information Aims and Scope Editorial Office Editorial Board Register as Author Register as Reviewer Instructions for Authors Abstracting and indexing Subscription Advertising Information Links
SCImago Journal & Country Rank

vol. 13
Case report

Intra-operative 3D hologram support with mixed reality technique based on CT-MRI fusion images: operation guidance for brain brachytherapy

Shifeng Liu
Hong Wang
Congxiao Wang
Wei Zhang
Gao Chen
Xiaokun Hu

Interventional Center, the Affiliated Hospital of Qingdao University, Qingdao, China
Department of Dermatology, Qingdao No. 6 People’s Hospital, Qingdao, China
He Wang Lan digital ST Co. Ltd., Hefei, China
J Contemp Brachytherapy 2021; 13, 2: 205–210
Online publish date: 2021/04/14
Article file
- Intra-operative 3D.pdf  [0.41 MB]
Get citation
JabRef, Mendeley
Papers, Reference Manager, RefWorks, Zotero


Permanent interstitial brachytherapy has been increasingly used and has resulted in good outcomes for primary or adjuvant treatment of brain tumors [1]. Interstitial brachytherapy with radioactive 125I seed implantation depends primarily on pre-operative planning and intra-operative optimization with brachytherapy treatment planning systems (TPSs) [2]. Both computed tomography (CT) and magnetic resonance imaging (MRI) have deficiencies in facilitating target delineation for brain tumors. Our previous research registered CT-MRI fusion images into brachytherapy TPS and three-dimensional printing individual template (3D-PIT) to guide brachytherapy for the treatment of brain gliomas and metastases, and achieved good results [3, 4].

In brain brachytherapy, there are several important steps involving pre-operative and intra-operative image acquisition and processing for recognizing the tumor location or vascular anatomy. Interventional surgeons and assistants performed seed implantation under guidance of pre-operative CT-MRI fusion data and intra-operative CT images. We hypothesize that the use of 3D visualization of inter-operative data with the use of a 3D support system would be helpful for interventional surgeons during brain brachytherapy.

Mixed reality (MR) technique combines the physical world with the digital world. Users of head-mounted displays (HMDs) intrinsically obtain a single-user point of view that frees their hands during brachytherapy. Using HMDs, interventional surgeons can see 3D computer graphics (CGs) based on CT-MRI fusion images with MR technique intra-operatively. The advantages of MR technique for surgical support are as follows: 1) no need for a sterilized monitor; 2) better visualization of the brain tumor than 2D images; and 3) ability to share 3D images among all interventional surgeons [5].

In this study, we report the first experience of brain brachytherapy performed under the guidance of an intra-operative 3D hologram, a CG model of the brain, with MR technique based on CT-MRI fusion images, and investigated the potential of 3D hologram as an intra-operative support tool.

Case report

We report a case of a patient with brain metastasis from primary esophageal cancer. Radical surgery for esophageal cancer was performed in February 2009. The brain metastasis detected by MRI in December 2018, was treated with a gamma knife. Four cycles of systemic chemotherapy were administered and ended in April 2019. 125I seeds were implanted in the brain metastasis with 3D hologram support with MR technique based on CT-MRI fusion images, as the patient’s metastasis relapsed, and the symptoms of dizziness and ataxia worsened in January 2020.

The patient underwent both CT (GE Light Speed 16 CT scanner, Milwaukee, WI, USA) and MRI (GE Signa HDx, Milwaukee, WI, USA) 3 days before brachytherapy. The slice thickness was 1.25 mm, and a vacuum pad was used to fix the body position. The positioning line and alignment reference line of the template were marked on the surface of the patient. Fusion of the pre-operative CT data and MRI images were performed using the method reported in our previous study [3]. The pre-operative plan was carried out on a TPS (KLSIRPS-3D, Beijing University of Aeronautics and Astronautics, Beijing, China) (Figure 1A-C).

Fig. 1

A) Pre-operative MRI showed the brain metastasis. B) Pre-operative planning design determined delineation of GTV (red line), needle tract (green line), organ at risk (OAR, yellow line), and calculated the dose distribution of target volume (isodose line) based on pre-operative CT-MRI fusion image by BTPS. C) Dose volume histogram of pre-operative plan


3D reconstruction of the brain was performed with human 3D visualization MR system (v1.0.0; ZWDX, China), using the above CT-MRI fusion image data. The brain skull, brain metastasis, main blood vessels, and ventricles were segmented from CT-MRI fusion data. 3D images of the brain were integrated. Polygon (STL) files were exported from the above-mentioned human 3D visualization MR system. After uploading the data, 3D images were automatically converted as a case-specific CG for MR, called a ‘3D hologram’. It takes a total of approximately 15 minutes from data upload until the generation of hologram. HoloLens (Microsoft Corporation, Redmond, WA, USA) is an optical see-through HMD that enables optical superposition of virtual content into the user’s direct view of the physical world. The data of hologram can be downloaded into HoloLens (Figure 2A, B).

Fig. 2

A) The operator adjusted the hologram through easy hand gesture. B) The hologram helped with brachytherapy guidance


Seed implantation was performed under general anesthesia. The patient was placed on the left side and fixed with a vacuum pad. All the interventional surgeons wore HoloLens devices. The hologram data were aligned to the brain of the patient. The hologram contributed to good visualization of the tumor location and peritumoral vessels. Then, all the interventional surgeons could confirm the tumor location and needle paths for the tumor just before 125I seed implantation, avoiding injuries to the brain vessels. This final confirmation of pre-operative CT-MRI fusion data with the intra-operative 3D hologram might lead to safe and accurate procedures. The implantation needle was used to puncture the tissue to the predetermined depth while avoiding the vessels. CT was performed during the puncture to verify the position of the implantation needle and seeds. The intra-operative CT was fused with the pre-operative MRI, and additional seeds were implanted if there were areas without seeds that could potentially lead to underdosing of the target. CT was performed during the seed implantation to observe the actual distribution of 125I seeds, until the actual distribution was basically consistent with that of the pre-operative plan (Figure 3A-C).

Fig. 3

A) Needle punctured the brain metastasis under the guidance of mixed reality technique. B) Post-operative dosimetry verification showed the actual seed distribution and calculated the dose distribution of target volume based on post-operative CT-MRI fusion image by BTPS. C) Dose volume histogram of post-operative plan


There were 6 implantation needle paths. The pre-operative PTV of the patient was 38.2 cm3. The activity of each single 125I seed used was about 0.7 mCi. The number of seeds implanted was 40. The prescribed dose was 120 Gy. The pre-operative and post-operative D90 were 132.0 Gy and 131.8 Gy, respectively. The pre-operative and post-operative V100 were 94.0% and 94.8%, respectively.

The patient had an uneventful course and was discharged home 3 days after the procedure. MRI was performed at 3 and 6 months after the seed implantation in a local hospital. The MRI data showed that tumor enhancement declined, necrosis existed in the tumor, and mass effect disappeared (Figure 4A, B). The symptoms of dizziness and ataxia significantly lessened.

Fig. 4

MRI data showing that tumor enhancement declined, necrosis existed in the tumor, and mass effect disappeared at 3 and 6 months after the procedure



In our study, the first experience of applying intra-operative 3D hologram support in brain brachytherapy was reported. For high-quality brachytherapy, care must be taken to avoid organs at risk, and in the case of the brain, it is extremely important to avoid the major vessels. The hologram was used for operative planning and to assess brain anatomy during 125I seed implantation to improve the safety of surgery. This hologram support was helpful for the procedure during brachytherapy.

The hologram support system used for this study was able to share the same images from every operator’s angle without monitors. The 3D hologram was also helpful for gaining good spatial awareness of the brain. HoloLens devices usually cannot be connected to the internet in operating rooms because of security problem. The HoloLens device of each interventional surgeon involved in seed implantation was configured in the same Wi-Fi network by a Wi-Fi hotspot. Such a Wi-Fi environment enabled sharing of the hologram. Furthermore, the operators could easily handle and manipulate the hologram with hand gestures. Though it differed greatly in individuals, it took some time for the interventional surgeons to get used to operating the HoloLens device with hand gestures.

Incekara et al. tested clinical applicability and accuracy of a wearable mixed-reality device for pre-operative neurosurgical tumor localization and surgical planning, and compared the device with standard neuro-navigation. They provided a proof of concept that the HoloLens had a potential for operative planning of brain tumor surgery with quantitative outcome measures [6]. Yu Saito et al. administered questionnaires assessing intra-operative hologram support in liver surgery and found that hologram scored significantly higher ‘physical demand’, and ‘effort’, and lower ‘performance’ than usual 2D support because it took surgeons a while to get used to operate HoloLens with hologram [7].

Teaching through holograms has advantages compared with traditional methods. Shao et al. evaluated teaching results in the form of objective questions and subjective examinations between hologram-based teaching and traditional mode teaching. They found that hologram technology might improve the quality of neurosurgical teaching, and this method should be promoted when teaching clinical subjects [8]. Hologram technology may later be used to assist clinical teaching.

Hologram technology is still in the early stages. First, the HMDs could be made to be lighter, simpler, and easier to operate. Second, if they can achieve real-time navigation during brachytherapy, these devices might be an ideal and comfortable instrument. Hologram has been introduced as an assisted guidance tool and not as a real-time navigation tool at this time. Third, further research is needed to explore this hypothesis.


This initial experience suggests that the use of intra-operative hologram with MR technique based on CT-MRI fusion images may help with brachytherapy guidance. Further research is needed to explore the potential of this approach in neuro-navigation.


The authors report no conflict of interest.



Chitti B, Goyal S, Sherman JH et al. The role of brachytherapy in the management of brain metastases: a systematic review. J Contemp Brachytherapy 2020; 12: 67-83.


Kudchadker RJ, Pugh TJ, Swanson DA et al. Preoperative treatment planning with intraoperative optimization can achieve consistent high-quality implants in prostate brachytherapy. Med Dosim 2012; 37: 387-390.


Gao Y, Han Y, Guo N et al. Value of CT-MRI fusion in iodine-125 brachytherapy for high-grade glioma. Oncotarget 2017; 8: 112883-112892.


Liu SF, Wang H, Wang CX et al. Dosimetry verification of 3D-printed individual template based on CT-MRI fusion for radioactive 125I seed implantation in recurrent high-grade gliomas. J Contemp Brachytherapy 2019; 11: 235-242.


Iannessi A, Marcy PY, Sugimoto M et al. A review of existing and potential computer user interfaces for modern radiology. Insights Imaging 2018; 9: 599-609.


Incekara F, Smits M, Dirven C et al. Clinical feasibility of a wearable mixed-reality device in neurosurgery. World Neurosurg 2018; 118: e422-e427.


Saito Y, Sugimoto M, Imura S et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg 2020; 271: e4-e7.


Shao X, Yuan Q, Qian D et al. Virtual reality technology for teaching neurosurgery of skull base tumor. BMC Med Educ 2020; 20: 3.

Copyright: © 2021 Termedia Sp. z o. o. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License (http://creativecommons.org/licenses/by-nc-sa/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material, provided the original work is properly cited and states its license.
Quick links
© 2021 Termedia Sp. z o.o. All rights reserved.
Developed by Bentus.
PayU - płatności internetowe