AR/VR in Healthcare
MRI Simulator App in VR
Overview:
The MRI simulator was my research project at Stanford University IMMERS lab, under Dr. Bruce Lewis Daniel and Dr. Christoph Leuze
The VR App features 4 sections for users to play. The 360 tour section, the mock MRI section, the breathing section, and the watch-a-movie section. The App was designed to prepare pediatric and claustrophobia patients for MRI examinations. I presented the work at IEEE-VR 2024.
Skills:
UI/UX Design, VR App Design, Unity 3D, C#, 3D Assets Creation, 3D Modeling, Shader Material, Unity Version Control
360 Tour
Breathing (anxiety-reducing meditation)
Main Mock MRI
Watch a Movie (10min hold still exercise)
(Recorded video was beta version; multiple features are updated)
AR Surgical Navigation of Guide-wire Placement for Pelvic Ring Fracture (ongoing)
Overview:
This is an ongoing project at JHU Laboratory for Computational Sensing + Robotics. The goal is to design a visual navigation system using the Microsoft Hololens 2 HMD to guide surgeons to accurately place guidewire for pelvic fracture surgeries. I designed the GUI and located the pivot point on the patient’s skin through point-cloud data acquired by the AHAT depth camera.
Skills:
MRTK, Hololens2 Research Mode API, OpenCV, Unity 3D, C#
GUI for tacking and placing the tool. Point cloud data was acquired from native depth camera on Hololens 2 HMD to find the pivot point on patient’s skin.
The automatic surgical process from making surface cut to final adjustments near bone.
VR in Education
Math Blaster VR Game
Overview:
I created an educational virtual reality game focused on mathematical calculations for Stanford Pediatric Hospital. Designed for users aged 10+, the game aims to enhance numerical awareness by providing practice in basic math facts. Similar to Beat Saber, players select a "number blaster" to slice the correct answers to addition, subtraction, division, and multiplication questions. Players can also choose from various locations such as a beach, forest, etc. The game features a range of math blaster options including a magic wand, giant pencil, lollipops, bananas, light saber, popsicles, and chicken drumsticks. These options are locked until players reach a new level, unlocking a new blaster.
Skills:
VR Game Development, Unity C#, UI/UX Design
Sample Scenes:
(App under beta testing; video will be updated.)
Radiology / Medicine
EEG Data Denoising
Overview:
To evaluate the combined effect of Transcranial Direct Current Stimulation (tDCS) and Therapeutical Instrumental Music Performance (TIMP) on corticobasal syndrome patients, I applied multiple data cleansing techniques to denoise the raw EEG data at the JHU Center for Music & Medicine.
Skills:
EEG Data Analysis, Denoising Techniques.
Distortion-free MRI Acquisition
Overview:
I had an opportunity to work with Dr. GUO, Hua at Tsinghua University to compare a distortion-free MRI acquisition technique (PSF-DWI) with traditional MRI sequences used at hospitals. This new approach may lead to better early disease diagnosis but needs a large-scale user study. I coordinated a large-scale data collection process by collaborating with local hospitals and successfully collected MRI raw data on 60 patients. The exam card parameters were fine-tuned for optimal image results on a 3T Phillips MRI machine. The data was used to train an AI model by Phillips Engineers. Potential future work includes MRI DICOM rendering in VR for better visualization, manipulation, and pre-operation planning. The work was accepted to ISMRM 2024.
Skills:
MRI Data Collection Pipeline, Exam Card Parameter Tuning
Reconstruction Results:
