iot ml lab
This is a lab course which will be presented at Advancing Computational Building Design 2019 in Chicago. The 3-hour course guides students through the creation of a simple IoT occupancy sensor by leveraging pre-trained Machine Learning models served by Tensorflow.js to detect entities in the space. The course also teaches some basic web design/development concepts, through the use of Vue.js to visualize and serve and the core business logic.
The project is structured as a basic web application in order to maximize user and device accessibility. Successful completion of the project would allow the user to “deploy” the project to any device with a camera, and access to a modern web browser.
Backend: Vue.js, Tensorflow.js
The first part of the course is focused around capturing the video feed and analyzing the stream using Tensorflow.js. On Glitch.com, parts 1 through 4 cover this section.
In order to create the web app, and capture the device’s video stream using the Navigator Web API, we use Vue.js. Vue is chosen because of the legibility and accessibility it affords to the development process. For an introductory lab course, it streamlines user interaction by abstracting away a number of tasks and interfaces the developer would otherwise need to address. At the end of this section, we have a fully functional webcam capture/analysis component which we can insert into any Vue app.
the data analysis model
The second part of the course is focused on creating a custom ML model to make predictions based on the data that our IoT sensor from the previous section has collected. On Glitch.com, parts 5 through 7 cover this section.
For the architecture of the custom ML model, we use a deep neural net created using brain.js. The brain.js library is yet another user-friendly, high-level implementation library. This allows us to focus less on the mathematical foundations behind machine learning, and instead focus on building the solution that will answer the questions we’re asking. Since the lab is only 3 hours, there is not much time for data collection, and isntead this section uses data provided by SpaceBot.ai from one of their deployed IoT sensors.