For many of us, our microwaves and dishwashers aren’t the first thing that come to mind when trying to glean health information, beyond that we should (maybe) lay off the Hot Pockets and empty the dishes in a timely way.
But we may soon be rethinking that, thanks to new research from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). The system, called “Sapple,” analyzes in-home appliance usage to better understand our health patterns, using just radio signals and a smart electricity meter.
Taking information from two in-home sensors, the new machine learning model examines use of everyday items like microwaves, stoves, and even hair dryers, and can detect where and when a particular appliance is being used.
For example, for an elderly person living alone, learning appliance usage patterns could help their health-care professionals understand their ability to perform various activities of daily living, with the goal of eventually helping advise on healthy patterns. These can include personal hygiene, dressing, eating, maintaining continence, and mobility.
“This system uses passive sensing data, and does not require people to change the way they live,” says MIT PhD student Chen-Yu Hsu, the lead author on a new paper about Sapple. “It has potential to improve things like energy saving and efficiency, give us a better understanding of the daily activities of seniors living alone, and provide insight into the behavioral analytics for smart environments.”
Of the two sensors, the “location sensor” uses radio signals to sense placement, and covers around 40 feet, or enough to cover a typical one-bedroom apartment. A user can walk around their apartment to set up the sensor, which allows it to understand the physical boundaries, and then the sensor can limit itself to that specified area.
The team says the system could potentially be useful during the Covid-19 pandemic, where there’s an increasing interest in contactless sensing of health and behaviors. They can imagine using passive sensor data to free up the need for caregivers to visit higher-risk populations and minimize overall in-person contact.
Sapple comes from the team’s growing body of research focused on using wireless sensing to better understand our complex human bodies — such as an in-body “GPS” sensor with the goal of tracking tumors or dispensing drugs, a wireless smart-home system for monitoring diseases and helping the elderly “age in place,” and another system for measuring gait to help monitor and diagnose various ailments.
Previous work in learning appliance usage has looked at using energy data from a utility meter. But this approach makes it challenging to tease out details, as the energy data is a mix of multiple appliances’ patterns all added together.
Unsupervised approaches — those in which training data aren’t labeled — assume patterns of individual appliances are unknown. However, since the utility meter measures the total energy used by the home, it’s really hard to learn individual appliances or detect them effectively.
Sapple stays in the unsupervised realm: It doesn’t assume we know the patterns of individual appliances, but instead uses data from a second sensor to help learn appliance usage patterns with self-supervision. For example, the location sensor captures a person’s motion as they approach a microwave, put food in it, and turn it on. The model then analyzes the data, and learns when specific appliances are turned on, and what their locations are in a home.
In addition to health, Sapple could potentially help reduce our heavy imprint on the natural world. By analyzing appliance usage patterns within homes, the system could be used to encourage energy-saving behaviors and improve forecasting and delivery for utility companies.
The team notes that their system’s approach solves some of the issues that can be tricky for in-home sensors. For example, using the location data doesn’t always imply appliance usage, as people can be next to an appliance without using it. Also, many appliances like refrigerators cycle their power and create “background events,” and there could be location data from multiple people in a home, but not all of them are related to appliance usage. Sapple solves these problems by learning when the two sensor streams become related, and uses that to discover when appliances are turned on, and their locations.
“As indoor location-sensing starts to potentially become as common as Wi-Fi in the future, the hope is that our technology can be effortlessly applied to all places with utility meters,” says Hsu. “This could enable new applications for passive health sensing in the homes. Utility companies, for example, could reduce peak demands by providing personalized feedback, optimize energy generation and delivery, and ultimately improve energy efficiency.”
Hsu wrote the paper alongside CSAIL PhD students Abbas Zeitoun and Guang-He Lee, as well as MIT professors Dina Katabi and Tommi Jaakkola. They presented the paper virtually at the International Conference on Learning Representations.