Digital twins have demonstrated great capabilities in the industrial setting, but the cost of building them prohibits their usage in home environments. We present HomeView, a system that automatically builds and maintains a digital twin using data from Augmented Reality (AR) headsets and Internet of Things (IoT) devices. We evaluated the system in a simulator and found it performs better than the baseline algorithm. The user feedback on programming IoT devices also suggests that contextual information rendered by HomeView is preferable to text descriptions.

DOI link: https://doi.org/10.1145/3526114.3558709

Citation: Zhuoyue Lyu, Jackie (Junrui) Yang, Monica S. Lam, and James A. Landay. 2022. HomeView: Automatically Building Smart Home Digital Twins With Augmented Reality Headsets. In The Adjunct Publication of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ‘22 Adjunct). Association for Computing Machinery, New York, NY, USA, Article 23, 1–6. https://doi.org/10.1145/3526114.3558709

Paper PDF: https://dl.acm.org/doi/pdf/10.1145/3526114.3558709

Code Repo: https://github.com/StanfordHCI/HomeView