Auptimize: Optimal Placement of Spatial Audio Cues for Extended Reality (ACM UIST 2024)

Authors: Hyunsung Cho, Alexander Wang, Divya Kartik, Emily Liying Xie, Yukang Yan, David Lindlbauer.

Auptimize is a spatial audio optimization system for Extended Reality (XR). It adjusts sound source positions relative to visual elements to compensate for human localization errors such as localization blur and front-back confusion, using integer programming on empirical perception data.

Find more details in our project page.

Auptimize Code

This repository consists of two components:

• python/ -- Optimization engine that builds perception models from participant data and solves for optimal sound positions using Gurobi integer programming.
• unity/ -- Demo VR application to showcase the experience with and without Auptimize on Meta Quest headsets.

Getting Started

Requirements

• Python 3.9 with numpy, pandas, scipy, gurobipy
• Unity 2021.3.9f1
• Meta Quest VR headset
• Gurobi license (free for academics: https://www.gurobi.com/solutions/licensing)

Quick Start: Demo

To try out Auptimize demo, open the Unity project at unity/ in Unity 2021.3.9f1. See unity/README.md for setup instructions.

Quick Start: Optimization

To try out the optimization process:

1. Set up Python environment:

   cd python
   conda create -n auptimize python=3.9
   conda activate auptimize
   python -m pip install numpy pandas scipy gurobipy

2. Generate Unity layout files from pre-computed results:

   python generate_unity_layouts.py

For running the full optimization pipeline or customizing layouts, see python/README.md.