HotSpot

Project introduction

HotSpot is a live COVID-19 location based contract tracing iOS mobile app that alerts users of confirmed cases in nearby retail stores using geofencing and predicts the number of COVID-19 cases on a given day using machine learning.

Issue/Inspiration

Digital COVID-19 contact tracing has not been very successful in the United States because conventional methods rely on person-person proximity detection. This trust system may, at best, alert a user days or potentially weeks after contact with an infected person. At this point, either the user has already experienced COVID-19 symptoms or the incubation period has safely passed. In the uncommon event that the user is an asymptomatic carrier, a retroactive notification is usually not enough stimulus to force users to get tested, which itself is not always accessible. This form of retroactive contact tracing does not prevent contact in the first place and can only work if used by a majority of the population. As such, our goal was to implement a location based contact tracing method to notify users by making attempts to detect spreading grounds for the virus and mark locations as crowded and infected.

What it does

We propose a contact tracing system in which locations (primarily retail stores) are tagged as infected rather than individuals and displayed on a map. This allows users to make a conscious choice to avoid certain locations, stopping contact with infections in the first place. The system also marks crowded locations in real time, which may be spreading grounds for the virus. Even if a handful of users are reporting infections, the entire population can benefit from this information and avoid infected locations. In order to alert users of infected and crowded locations, we implemented a geofencing feature which will send push notifications to users when they accidently enter such a location. We protect user privacy by using Radar.io, Google Identity Platform, and a secure Cloud Firestore database. Furthermore, we implemented a Support Vector Regression Model and Artificial Neural Network to predict the number of COVID cases on a given day with high performance.

How did your project evolve with the support of the COVID-19 hackathon fund by Google Cloud?

Under the guidance of our mentor Haniel Diaz, we were able to set effective project goals and regularly present our work to gather progress feedback from the eyes of a user. With the assistance of Haniel, we were able to implement a fully-functioning geofencing feature. Previously, users had to constantly check the map to learn about infected/crowded locations. With geofencing, we eliminated the need for checking by providing live notifications to users when entering an infected zone. We made sure to continue our goal of protecting privacy by performing location calculations locally with Radar.io instead of sending data to the web.

For the back-end database, we were able to validate the user’s identity by having login and signup Authentication with Google OAuth instead of storing raw passwords with a naive, insecure sign-in. This also solves a privacy concern as the password is hashed using bcrypt algorithm. We also reconfigured the Cloud Firestore Database to store documents pertaining to each user’s state.

In terms of the User Interface of the iOS Mobile Application, we worked on enhancing the transitions between each Storyboard Scene Kit View Controller. We added custom animations in the home screen, creating and prototyping designs using wireframing techniques to make the application more appealing to the user. We also designed a logo for the application, making the application more presentable and user friendly.

With our knowledge of Python, we decided to tackle creating a choropleth heatmap of COVID-19 cases for each country pertaining to the user’s state. The Python Script read through The New York Times COVID-19 cases per country based on the FIPS code and created a visual heat map using plotting libraries. This can better assist users as they are traveling to avoid travelling into counties with greater COVID-19 cases, avoiding traffic.

Finally, we trained a couple machine learning models to predict the number of COVID cases on a given day. Our first attempt was using an Artificial Neural Network along with the JHU COVID-19 dataset. Although this architecture was much more general, since the infection trend of any geolocation could be predicted, the correlation coefficient was about 0.66. To achieve better performance, we switched to a Support Vector Regression model that had to be trained on a per-county basis, but was able to predict cases with a correlation coefficient of over 0.99.

The guidance of Google Cloud and the resources provided were a valuable part of all of the new developments made on the project since our initial hackathon.

How you built it

The iOS application was written in the Swift programming language with the Xcode IDE. A variety of Apple and external frameworks were incorporated into the application through the Cocoapods dependency manager. The screens of the app were created with UIKit, specifically UIViewController.

After opening the application the user is welcomed with an Introduction walkthrough screen with a custom animation of a radar tracking. Next, the user has the ability to either login or signup using Google Authentication SignIn Methods with Google Identity Platform. With this approach, we use Google OAuth, an internet protocol allowing 3rd parties to exchange data without the user actually signing into an account. This also avoids having to use the user’s password in any way since the bcrypt password hashing algorithm takes care of this. By signing in without any account, Google SignIn can authenticate the for us, and internally query the user email into Google Firestore. Inside the application, a tab bar menu controller makes it intuitive for users to explore parts of the app and transition to different screens. Overall the experience is user friendly, allowing the app to be widely distributed.

Location tracking was implemented with the Radar.io software development kit, which is built on top of the CoreLocation framework. In order to protect user privacy while tracking location, we only start tracking once a patient has moved a certain distance, so that tracking only begins after leaving home. Every few seconds, the application searches for nearby retail stores and checks if the user has stayed in a retail store for a certain amount of time. Confirmed store visits are stored in a Cloud Firestore under the corresponding user.

Once a user wants to report a COVID infection, the Firestore database is queried for the user’s store visits, and the user is asked to confirm the visits. The confirmed infection sites are shown on a map built with MapKit. If a specific location has a large number of people, the Firestore database marks it as “crowded”, and the UIMapView displays a yellow warning circle. The UIMapView is refreshed whenever a user opens it according to the latest Firestore queries.

Geofencing was implemented with a conjunction of Apple’s CoreLocation and Radar.io for location tracking. Once location tracking has determined that the user is in a crowded or infected zone, a push notification is sent from the app, which was created with the NotificationCenter framework.

To predict the number of COVID cases in a location on a given day, we trained a Support Vector Regression model using Pytorch. We used the Johns Hopkins COVID-19 dataset to acquire data for various California Counties and train the model. The input feature is the number of days since first infection. The output of the model is the number of cases on the day specified by the input. We used a Radial Basis Function kernel and a regularization penalty of 1e4 to train the SVR. Our results were amazing, and we were able to predict the number of cases for a specific county with a correlation coefficient of over 0.99. The drawback to this approach was that the model has to be trained on certain counties at a time. We tried to make the model more general by using an Artificial Neural Net with geolocation as an input feature as well. We trained the neural net on the entire California dataset and were able to get a correlation coefficient of 0.66.

We are using Cloud Firestore as a database to document store visits for each user. If a user reports that they are infected, or have COVID-19 symptoms, the app retrieves visited store locations from Firestore for the user to confirm. After confirmation, a document of the location is made on Firestore to track infection status, number of visitors (live), and other variables.

Challenges you ran into

The machine learning features were hard to implement due to the difficulty of training high performance models on limited hardware. Our initial attempt at COVID case prediction was a neural network, but we weren’t able to achieve a high degree of accuracy. We overcame this issue by trying a different approach with Support Vector Regression. Although SVR did not work well with multiple locations, we found that training one SVR per location produced a very effective result.

Another challenge was ensuring all location and database updates were happening live and without lag. Since the response time for a Firestore query varies and can be slow at times, we have to ensure that UI updates only happened once a response was received. With map updates, Radar.io tracking, Firebase queries, and geolocation all happening at the same time, it was hard to manage so many background threads. We had to read up on multithreading with iOS to coordinate all the live updates. With the database, storing Firebase User OAuth emails queries and firebase login queries was another challenge. Normally people use a naive login system or a Google Identity Platform SignIn Method, but having the Firebase Firestore query information was much harder. We accepted our challenge and figured out a way to update the document inside Firestore to allow both login methods to successfully work.

We couldn't test our app with legitimate location updates due to COVID and infeasibility of so much travel. Our solution was to contrive test cases using .gpx files to simulate location. We created our own gpx files with geolocation for various stores to simulate user visits and reports.

To run the python scripts in the iOS mobile application was another challenge to tackle. Normally Python scripts run once the program has been launched, and terminate afterwards, but showing this in an iOS application is tricky. We decided that using a WKWebView in the Swift iOS app would be the best solution, since transitioning the python scripts into a website and making that website link activated on the iOS app could allow that python program to be available for users. Running a python program via Flask is feasible, but we found it much easier to run the local host html file via GitHub Pages to allow a public website link. This link was then inserted into the WKWebView interface allowing the app to directly access the link running the python choropleth heatmap.

Accomplishments you are proud of

Over the course of the project, we’ve read countless documentation and worked with a number of frameworks and libraries. It was very fulfilling to build an entire application from start to end, while also trying to follow best practices in industry. In terms of technical skills, we are proud to have learned how to use Google Cloud products such as Cloud Firestore and Identification platform.

What you learned

One of the most important skills we learned was connecting a secure backend database to our iOS application. It was interesting to learn how to query databases and publish information. Another fascinating skill we learned was machine learning. At the start of the project, we had very limited knowledge of machine learning. However, we read tutorials and implemented various models in Pytorch to become familiar with the different use cases of models. Now, we have a working knowledge of multiple ML models and how to apply them. Working on HotSpot has been a great way to begin diving into machine learning with practical applications.

In terms of managing large-scale projects, we learnt to configure GitHub Organizations and manage workflows similar to working in an industry setting. Collaborating with others about the development of multiple codebases was helpful to understand the bigger picture.

What’s next for your project

Our current goal is to wrap up any loose ends and publish our application on mobile app stores. COVID-19 is still very much a threat, so it would be great if we could amass a user base to help combat infection spread. Also, we plan to write a research paper on HotSpot and make our codebase open source for any others to build on. Making our code publically available and licensed under terms and conditions of use, we can have others contribute to our project. We hope that our lengthy experience in building this mobile application will enable others to continue the fight against COVID-19.

What Google Cloud products did you use to build your project?

• Google Cloud Firestore (query user data)
• Google Firebase Authentication (login methods)
• CocoaPods Dependency Manager (Configure Firebase into iOS) 
• Google Identity Platform
  • Google OAuth (Authorization Protocol)
  • Google SignIn (Google Account)