Lab  The course is hosted entirely on Glitch.com, eliminating the need for local dependency management and allowing students to jump right into creation. The lab focuses on building a personal IoT occupancy sensor, using Tensroflow.js pre-trained models for analysis, and Vue.js for visualization.

Lab The course is hosted entirely on Glitch.com, eliminating the need for local dependency management and allowing students to jump right into creation. The lab focuses on building a personal IoT occupancy sensor, using Tensroflow.js pre-trained models for analysis, and Vue.js for visualization.

iot ml lab

This is a lab course which will be presented at Advancing Computational Building Design 2019 in Chicago. The 3-hour course guides students through the creation of a simple IoT occupancy sensor by leveraging pre-trained Machine Learning models served by Tensorflow.js to detect entities in the space. The course also teaches some basic web design/development concepts, through the use of Vue.js to visualize and serve and the core business logic.

The project is structured as a basic web application in order to maximize user and device accessibility. Successful completion of the project would allow the user to “deploy” the project to any device with a camera, and access to a modern web browser.

Tools:
Backend: Vue.js, Tensorflow.js
Deployment: Glitch.com

Stack Vue.js is used to create a web page with a video capture using the “Navigator” Web API. After a device’s video feed is captured, the stream data is passed to Tensorflow.js, where the PoseNet and CocoSSD models are used to determine whether people are occupying the space in the video feed.

the sensor

The first part of the course is focused around capturing the video feed and analyzing the stream using Tensorflow.js. On Glitch.com, parts 1 through 4 cover this section.

In order to create the web app, and capture the device’s video stream using the Navigator Web API, we use Vue.js. Vue is chosen because of the legibility and accessibility it affords to the development process. For an introductory lab course, it streamlines user interaction by abstracting away a number of tasks and interfaces the developer would otherwise need to address. At the end of this section, we have a fully functional webcam capture/analysis component which we can insert into any Vue app.

Analysis Component The Vue.js component captures the webcam’s video stream, and pipes the data to the Tensorflow.js instance which computes whether or not a person is visible in a given frame within the stream.


the data analysis model

The second part of the course is focused on creating a custom ML model to make predictions based on the data that our IoT sensor from the previous section has collected. On Glitch.com, parts 5 through 7 cover this section.

For the architecture of the custom ML model, we use a deep neural net created using brain.js. The brain.js library is yet another user-friendly, high-level implementation library. This allows us to focus less on the mathematical foundations behind machine learning, and instead focus on building the solution that will answer the questions we’re asking. Since the lab is only 3 hours, there is not much time for data collection, and isntead this section uses data provided by SpaceBot.ai from one of their deployed IoT sensors.