Completed Projects

An increasingly large number and variety of sensors are collecting and transmitting data about many of the smallest actions in our lives. Often, these data are used by companies to try to sell us things. But they are also increasingly being used to help us with tasks in our lives. Sensor data from our phones is collected to help us choose driving routes that avoid traffic. Step-counting sensors try to help us stay active. The aim of these studies is to learn how to use these kinds of sensor data to improve the lives of people with mental health problems. The personal sensing research at the Center for Behavioral Intervention Technologies harnesses sensor data collected from mobile phones and other devices to identify behaviors and states related to mental health. These algorithms can then be used to improve the quality and effectiveness of digital mental health interventions such as apps and bots.

Studies targeted the general population and people with symptoms of depression. As a result, we published a number of papers showing that mobile phone sensor data can be used to identify people who may be at greater risk for depression, as well as more specific behaviors such as when people go to bed and get up.

The PIs were David C. Mohr, PhD, and Konrad Kording, PhD, with funding from NIMH grant R01-MH111610.

Resulting Publications

• Mohr DC, Zhang M, Schueller SM. Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. Annu Rev Clin Psychol. 2017 May 08;13:23-47. PMID: 28375728. doi: 10.1146/annurev-clinpsy-032816-044949.
• Saeb S, Cybulski TR, Schueller SM, Kording KP, Mohr DC. Scalable Passive Sleep Monitoring Using Mobile Phones: Opportunities and Obstacles. J Med Internet Res. 2017 Apr 18;19(4):e118. PMID: 28420605. doi: 10.2196/jmir.6821.
• Saeb S, Lattie E, Schueller SM, Kording K, Mohr DC. The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ. 2016;4(e2537). doi: 10.7717/peerj.2537.
• Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, et al. Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study. J Med Internet Res. 2015;17(7):e175. PMID: 26180009. doi: 10.2196/jmir.4273.
• Liu, T., Nicholas, J., Theilig, M.M., Guntuku, S., Kording, K., Mohr, D.C., Unger, L. (2019) Machine learning for phone-based relationship estimation: The need to consider population heterogeneity. Wearable Ubiquitous Technologies, 3, Article 145.
• Nicholas J, Shilton K, Schueller SM, Gray EL, Kwasny MJ, Mohr DC. The Role of Data Type and Recipient in Individuals' Perspectives on Sharing Passively Collected Smartphone Data for Mental Health: Cross-Sectional Questionnaire Study. JMIR Mhealth Uhealth. 2019;7(4):e12578.
• Meyerhoff, J. Liu, T. , Kording, K.P., Ungar, L.H., Kaiser, S.M., Karr, C.J., Mohr, D.C. (under review) Evaluation of changes in depression, anxiety, and social anxiety using smartphone sensor features: Longitudinal cohort study. JMIR Mental Health.

IntelliCare is a suite of multiple smartphone apps, each of which provides a microintervention targeting a single behavioral strategy for depression and/or anxiety. Rather than using a single intervention tool to target a disorder, which can be time-consuming for the user, we have used a platform approach, in which a variety of apps, each targeting a singular psychological strategy. This is particularly important for mobile apps, which typically have one objective and are quick and easy to use. IntelliCare apps are designed to fit into the fabric of users’ lives. They are simple and quick to use, most requiring less than 30 seconds for each use. The user can select those apps that are most useful and ignore those that do not meet their needs. The apps are available on Google Play and Apple App Store.

The IntelliCare Hub app, if installed on the phone, orchestrates the user’s experience by making weekly recommendations for new apps to try. However, while the recommendation is to look at the app, the user is encouraged to those that are helpful and is free to ignore those that are not.

IntelliCare can be used alone or in conjunction with human coaching. Coaches are provided with an online dashboard that provides visibility into the user’s activity on IntelliCare apps, tracks symptom severity and manages text messaging communications.

IntelliCare development has been moved to a startup out of Northwestern University, Adaptive Health, which is working to disseminate and implement IntelliCare broadly. In this effort, IntelliCare is being deployed system-wide in the Rush University Medical Center’s Collaborative Care Program, where it has been fully integrated into their Epic EHR.

• Project Names: 
  • Artificial Intelligence in a Mobile Intervention for Depression (IntelliCare Study Randomized Control Trial)
  • Novel Methods for Evaluation and Implementation of Behavioral Intervention Technologies for Depression
  • Implementing an innovative suite of mobile applications for depression and anxiety
• PI: David C. Mohr, PhD
• Funding: NIMH grants R44-MH114725, R01-MH110482 and R01-MH109496

Resulting Publications

• Graham AK, Greene CJ, Kwasny MJ, M.J., Kaiser, S.M., Lieponis, P., Powell, T., Mohr, D.C. Coached Mobile App Platform for the Treatment of Depression and Anxiety Among Primary Care Patients: A Randomized Clinical Trial. JAMA Psychiatry. 2020.
• Mohr DC, Schueller SM, Tomasino KN, Palac, H., Kwasny, M.J., Weingardt, K., Karr, C.J., Kaiser, S.M., Rossom, R., Bardsley, L.R., Caccamo, L., Stiles-Shields, C., Schueller, S.M. Comparison of the Effects of Coaching and Receipt of App Recommendations on Depression, Anxiety, and Engagement in the IntelliCare Platform: Factorial Randomized Controlled Trial. J Med Internet Res. 2019;21(8):e13609.
• Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety. J Med Internet Res. 2017;19(1):e10
• Lattie EG, Schueller SM, Sargent E, et al. Uptake and Usage of IntelliCare: A Publicly Available Suite of Mental Health and Well-Being Apps. Internet Interv. 2016;4(2):152-158.
• Kwasny MJ, Schueller SM, Lattie E, Gray EL, Mohr DC. Exploring the Use of Multiple Mental Health Apps Within a Platform: Secondary Analysis of the IntelliCare Field Trial. JMIR Ment Health. 2019;6(3):e11572.
• Cheung K, Ling W, Karr CJ, Weingardt K, Schueller SM, Mohr DC. Evaluation of a recommender app for apps for the treatment of depression and anxiety: an analysis of longitudinal user engagement. J Am Med Inform Assoc. 2018;25(8):955-962.

IntelliCare for College Students is an app-based program to help university students prioritize self-care and connect them with appropriate mental health and wellness resources. This program was developed as part of a collaboration between Northwestern University, the University of Illinois at Chicago and Northern Illinois University. It built upon the IntelliCare platform of apps and was developed in conjunction with stakeholders from the university student communities of interest. Once developed, this program was tested in a single-arm pilot study and is currently being rolled out more broadly on both campuses. The impact of the program on student mental health and college counseling center utilization will be examined. 

The PI was Emily Lattie, PhD, with funding from NIMH grant K08MH112878.

Resulting Publications

• Lattie, E.G., Cohen, K. A., Winquist, N., & Mohr, D.C. (2020). Examining an app-based mental health self-care program, IntelliCare for College Students: A single-arm pilot study. JMIR Mental Health, 7, e21075.
• Cohen, K.A., Graham, A.K. & Lattie, E.G. (2020). Aligning students and counseling centers on student mental health needs and treatment resources. Journal of American College Health. DOI: 10.1080/07448481.2020.1762611
• Lattie, E.G., Kornfield, R., Ringland, K., Zhang, R., Winquist, N., & Reddy, M. (2020). Designing mental health technologies that support the social ecosystem of college students. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-15.

An increasingly large number and variety of sensors are collecting and transmitting data about many of the smallest actions in our lives. Often, these data are used by companies to try to sell us things. But they are also increasingly being used to help us with tasks in our lives. Sensor data from our phones is collected to help us choose driving routes that avoid traffic. Step-counting sensors try to help us stay active. The aim of these studies is to learn how to use these kinds of sensor data to improve the lives of people with mental health problems. The personal sensing research at the Center for Behavioral Intervention Technologies harnesses sensor data collected from mobile phones and other devices to identify behaviors and states related to mental health. These algorithms can then be used to improve the quality and effectiveness of digital mental health interventions such as apps and bots.

Studies targeted the general population and people with symptoms of depression. As a result, we published a number of papers showing that mobile phone sensor data can be used to identify people who may be at greater risk for depression, as well as more specific behaviors such as when people go to bed and get up.

The PIs were David C. Mohr, PhD, and Konrad Kording, PhD, and funding was provided by NIMH grant R01-MH111610.

Resulting Publications

• Mohr DC, Zhang M, Schueller SM. Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. Annu Rev Clin Psychol. 2017 May 08;13:23-47. PMID: 28375728. doi: 10.1146/annurev-clinpsy-032816-044949.
• Saeb S, Cybulski TR, Schueller SM, Kording KP, Mohr DC. Scalable Passive Sleep Monitoring Using Mobile Phones: Opportunities and Obstacles. J Med Internet Res. 2017 Apr 18;19(4):e118. PMID: 28420605. doi: 10.2196/jmir.6821.
• Saeb S, Lattie E, Schueller SM, Kording K, Mohr DC. The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ. 2016;4(e2537). doi: 10.7717/peerj.2537.
• Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, et al. Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study. J Med Internet Res. 2015;17(7):e175. PMID: 26180009. doi: 10.2196/jmir.4273.
• Liu, T., Nicholas, J., Theilig, M.M., Guntuku, S., Kording, K., Mohr, D.C., Unger, L. (2019) Machine learning for phone-based relationship estimation: The need to consider population heterogeneity. Wearable Ubiquitous Technologies, 3, Article 145.
• Nicholas J, Shilton K, Schueller SM, Gray EL, Kwasny MJ, Mohr DC. The Role of Data Type and Recipient in Individuals' Perspectives on Sharing Passively Collected Smartphone Data for Mental Health: Cross-Sectional Questionnaire Study. JMIR Mhealth Uhealth. 2019;7(4):e12578.
• Meyerhoff, J. Liu, T. , Kording, K.P., Ungar, L.H., Kaiser, S.M., Karr, C.J., Mohr, D.C. (under review) Evaluation of changes in depression, anxiety, and social anxiety using smartphone sensor features: Longitudinal cohort study. JMIR Mental Health.

With support from Microsoft's AI for Accessibility program, the Center for Behavioral Intervention Technologies is working to develop a digital mental health intervention that can help young adults learn to manage their mental health concerns. Even though mental health concerns like depression and anxiety are extremely common among young adults ages 18-25, most young adults are interested in managing these concerns on their own, without formal treatments like medication or psychotherapy. Many are open to using digital tools to help them with their mental health. However, because young people have little contact with the mental health system and can’t easily pay for digital health tools, there are few digital mental health solutions for this population.

Through our user-centered design process, we are working directly with young people with mental health concerns to understand the challenges they are facing and how technology can help. Our intervention will be based around text messaging, the most frequently used application on the phone. Our approach will integrate a large bank of interactions and content that can be personalized, through machine learning, to meet the diverse needs of young people. Over time, the intervention will learn which psychological intervention strategies and styles of interaction are most acceptable and engaging for each user, helping to address behaviors, thinking patterns and social challenges.

We are partnering on this project with computer scientists at the University of Toronto and Mental Health America, our nation’s largest mental health advocacy organization. In 2015, Mental Health America placed mental health screening tools on their website that are now used by more than one million visitors annually, and about 5 million visitors completing self-screeners in 2021. The intervention will be developed and evaluated with adults with depression, and young adults who visit the Mental Health America screening website.

• Project Name: Personalized Messaging Intervention for Young Non-Treatment Seeking Adults Delivered as a Public Health Service
• PIs: Rachel Kornfield, PhD and David C. Mohr, PhD
• Funding: Microsoft’s AI for Accessibility Program, K01-MH125172

Stepped care is a model of treatment delivery in which patients are first provided a low-intensity treatment, are systematically monitored for progress and are stepped up to more intensive treatments if they fail to show adequate improvement. In this project, the stepped care program initiated treatment for depression with ThinkFeelDo, our web-based digital mental health treatment, and stepped up those who did not meet improvement criteria to telephone-administered cognitive behavioral therapy (tCBT). This trial found that stepped care was approximately half as costly to deliver but no less effective than tCBT.

The PI was David C. Mohr, PhD, with funding from NIMH grant R01-MH095753.

Associated Clinical Trial

• Stepped Telemental Health Care Intervention for Depression

Resulting Publications

• Mohr, D.C., Lattie, E.G., Tomasino, K.N., Kwasny, M.J., Kaiser, S.M., Gray, E.L., Alam, N. Jordan, N., Schueller, S.M. (2019) A randomized noninferiority trial evaluating remotely-delivered stepped care for depression using internet cognitive behavioral therapy (CBT) and telephone CBT. Behaviour Research and Therapy, 123, 103485
• Nicholas, J., Ringland, K.E., Graham, A.K., Knapp, A.A., Lattie, E.G., Kwasny, M.J., Mohr, D.C. (2019) Stepping up: Predictors of “stepping” with in an iCBT stepped-care intervention for depression. International Journal of Environmental and Public Health, 4689.
• Nicholas, J., Knapp, A.A., Vergara, J.L., Graham, A., Gray, E.L., Lattie, E.G., Kwasny, M.J., Mohr, D.C. (in press) A brief head-to-head non-inferiority comparison of an internet-based and telephone-delivered CBT intervention for adults with depression. Journal of Affective Disorders,
• Graham, A., Nicholas, J., Gray, E., Knapp, A.A., Lattie, E.G., Kaiser, S.M., Kwasny, M.J., Mohr, D.C., (under review) Trajectories of symptom change over the course of remotely-delivered treatment for depression, Behaviour Research and Therapy.

ThinkFeelDo is a web-based treatment for depression designed to extend treatment to those who may be unable to access traditional psychological interventions.

The ThinkFeelDo web application is based on the principles of cognitive behavioral therapy and is accessible via desktop and laptop computers as well as mobile devices. ThinkFeelDo provides information via interactive skill-building exercises, text and video, and supports communication with a human coach via secure messaging. Coaches use an online dashboard to see a user’s activity on the site, allowing for targeted positive reinforcement, assistance and support.

Through our clinical trials, ThinkFeelDo has been tailored to meet the needs of diverse populations across the age spectrum, including adolescents, adults and older adults. Variations of ThinkFeelDo have also included social network platforms for users to engage with one another for added social support. ThinkFeelDo has been shown to be effective for depression in general populations, as well as for youth and older adults.

The PI was David C. Mohr, PhD, with funding from NIMH grants R01-MH095753 and P20-MH090318.

Associated Clinical Trials

• Stepped Telemental Health Care Intervention for Depression
• Technology Assisted Intervention for the Treatment and Prevention of Depression
• Technology Assisted Intervention for the Treatment and Prevention of Depression

Resulting Publications

• Mohr DC, Cuijpers P, Lehman K. Supportive Accountability: A Model for Providing Human Support to Enhance Adherence to eHealth Interventions. J Med Internet Res. 2011;13(1):e30.
• Schueller SM, Tomasino KN, Mohr DC. Integrating Human Support into Behavioral Intervention Technologies: The Efficiency Model of Support. Clinical Psychology: Science and Practice. 2016(24):27-45.
• Tomasino KN, Lattie EG, Ho J, Palac HL, Kaiser SM, Mohr DC. Harnessing Peer Support in an Online Intervention for Older Adults with Depression. The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry. 2017;25(10):1109-1119
• Lattie EG, Ho J, Sargent E, Tomasino, K.N., Smith, J.D., Brown, C.H., Mohr, D.C. Teens engaged in collaborative health: The feasibility and acceptability of an online skill-building intervention for adolescents at risk for depression. Internet Interventions. 2017;8:15-26.
• Chen AT, Wu S, Tomasino KN, Lattie EG, Mohr DC. A multi-faceted approach to characterizing user behavior and experience in a digital mental health intervention. J Biomed Inform. 2019;94:103187.