Remind A.I.

Project introduction

COVID-19 has caused many institutions and businesses to transition their staff to a remote setting. However, as a result of this, students and workers are now required to be in front of a computer almost all the time. Many people need to be on a computer to complete assignments and projects, which require them to sit for hours a day. Unfortunately, having to sit for so long can lead to many physical health issues. These can range from many issues such as neck strains, back pain, and knee pain. Sources such as the National Health Service of UK, and PainScience, have recommended a maximum of 30 minutes of sitting at one go. However, this number is always exceeded. To solve this issue, I decided to develop a web application that utilizes deep learning to track if users are sitting or standing. If the application detects that the user has been sitting for more than the recommended time, an alarm or notification will be activated to notify the user to stand. For reinforcement, this alert will continually be activated until the app detects the user has stood up. The overall goal of this is to prevent users from sitting for prolonged periods of time.

Issue/Inspiration

With all activities and jobs becoming remote, the amount of time people spend on their computers has increased exponentially. Unfortunately, this means people need to spend hours sitting down on their computers. This can lead to many physical health issues such as back pain, and neck strain. According to the National Health Service of UK, it is recommended to only sit for a maximum of 30 minutes at a time, yet, this number is exceeded all the time. To solve this, I decided to implement a reminder application that utilizes state-of-the-art deep learning technology to track how long a user has been sitting. If the user exceeds the limit, the application will reinforce the user to stand.

What it does

Remind A.I. takes snapshots from the user’s webcam every 5 seconds and runs the image through an image classification model to predict if they’re sitting or standing. The web app utilizes this prediction to determine if a user has been sitting for more than the recommended 30 minutes. If the user exceeds the time recommended to be sitting down, it will either output a web notification or sound off an alarm to notify the user that it is time to stand up and take a break.

How did your project evolve with the support of the COVID-19 hackathon fund by Google Cloud?

With the help of the COVID-19 hackathon fund, I was able to continue the process of the application. The fund allowed me to run the models on a backend server and maintain the use of Firebase as an authentication system.

How you built it

To build this application, I utilized ReactJS for the front end, Python with Tensorflow for the model development, and Python with Flask for the backend. For the front end, I decided to use the ReactJS framework, which includes the implementation of Firebase of the application, with additional HTML and CSS styling.

For the development of the image classification model, I decided to use Tensorflow and Keras in Python. I collected over 5,000 images of myself and friends of mine that contain images of them sitting and standing. I then partitioned the images to create a binary image classification model. Currently, the model achieves an accuracy of 93.4% on the testing data.

For the backend, I decided on utilizing Python with Flask. I implemented an API that takes in snapshots from the user’s webcam as input, then processes the image to classify if the user is sitting or standing. The output of this API is then finally returned to the front-end application for tracking.

Challenges you ran into

Many challenges were tackled during the process of developing the application. One of the main challenges faced was developing the API on Flask. There were issues that arose with the utilization of CORS when connecting to the frontend and backend. However, after many attempts of debugging and research on documentation, this challenge was overcome.

Accomplishments you are proud of

I’m very proud of the final development of the application. I’m proud I was able to develop an application that can help people prevent physical health issues.

What you learned

I learned that patience is important when developing applications. It’s important to have a plan ready before tackling the issue. This can help in higher productivity and result in a more effective project.

What’s next for your project

For the next step, I hope to deploy this application, so others around the world can utilize this to their advantage.

What Google Cloud products did you use to build your project?

For this project, I used Firebase and the Cloud Firestore.