Concept The workflow aims to overlay building information and other data on top of the physical space which is currently under construction.


This project was developed at the 2019 AEC Hackathon in Seattle. It won First Place in the competition under the “Best Overall” project category.

This is a project/workflow for using Augmented Reality on a construction site to overlay information on top of the in-progress building. The workflow allows for the user to overlay data, 3D geometry, and even interact with the information by creating issues, markups, and other forms of comments. That data is then pushed to a database in real-time, and visualized in a web app - allowing everyone back in the office to see what the person on site is observing.

Project GitHub (AR Module)
Project GitHub (Web App)
Project GitHub (API Backend)

Backend: Unity, Vue.js, Charts.js, Express. C#, Javascript

Team: Mark Kinsman, Simon Manning, Petr Mitev, Tito Pagan, Logan Smith
Presentation: Link

ar markups

The first part of the workflow happens at the construction site while wearing a HoloLens VR headset. The HoloLens app we developed allows the user to overlay building information and data such as parameters, and 3D geometry on top of the elements that are currently under construction all around the user. This allows them to check the progress of the physical space against the conditions defined by the Architects and Engineers in the digital space. To capture their review, the HoloLens app offers an interface for “tapping” on objects around you to add comments, images, questions, or other forms of markups.

Interface & Unity

The AR portion of the workflow was developed in Unity, using Vuforia. Building to the HoloLens proved to be a challenge, even with representatives from Microsoft at the event trying to troubleshoot the build. Luckily, the solution did eventually compile. To add some interactivity and visual appeal to the AR, we also created animated interface icons - compressing the amount of information that would otherwise clutter the AR space into an accessible and engaging interface.

AR Interface The interface for the Augmented Reality was modeled and animated to condense options and information into a tidy and compact package that doesn’t clutter the user’s vision.

Model The BIM model is used as the base/overlay in Unity to provide element parameters and metadata on top of the built environment.

web interface

The last portion of the workflow is the web portion. As a user is submitting comments and reviews on the job site through the HoloLens, that data is pushed to a Firebase Realtime Database. When the data in the database changes, the web app reflects those changes as it is constantly listening to them. That means that any time the web page is opened, the latest data is being visualized, without any artifacts from previous reviews. The web app allows colleagues or partners of the person on the job site to keep up with their review, and start addressing any critical items from the office.