Since U Been Gone: Augmenting Context-Aware Transcriptions for Re-Engaging in Immersive VR Meetings

Overview:

I have co-designed and independently programmed an AI co-pilot in VR, a multiplayer conference system with live transcription and AI-summary features to help participants keep up with the conversation. The goal is to improve information recall through a gaze-aware, context-aware intelligent summarization system. This system I built may also benefit people with hearing disabilities. This work was accepted to CHI 2025 as full paper.

Why Important?

This system effectively helps meeting participants keep up with the conversation, which is beneficial for an active discussion. Similar systems can be designed in AR HMD for face-to-face meetings.

Skills:

Study Design, Unity 3D, Photon PUN & Fusion & Voice, Meta Avatar SDK, Prompt Engineering, API Calling