Consumer Health Application Review

Health Mate

Name: Shehab Sennah

Student Number: 999854910

Date: February 7, 2017

Course: MHI2003

Introduction:

76% of Canadians owned a smartphone in 2016, a 38% increase from 2014¹. Extensive proliferation of these devices provides an avenue for broadening healthcare into the mobile world. This can be achieved using consumer health apps (short for applications). Consumer health apps are software that can be downloaded (sometimes with a fee) on various operating systems. More than 165,000 health apps are available, geared towards all aspects of health and medical treatment. They can utilize sensors, GPS, computer processing and the internet to benefit the health of patients². These apps also play a role in the global need to make healthcare more affordable, efficient and comprehencive³. eHealth Ontario has decided that it will be endorsing consumer health apps. However, recommending an app to patients can have some serious consequences if it has inaccuracies or inflicts harm⁴. This report suggests a method to determine which apps should be endorsed by eHealth Ontario.

eHealth Endorsement

This report proposes 5 main criteria for the evaluation of apps to be endorsed by eHealth Ontario. These criteria are broken down into components, each to be rated individually with a score of 0 to 4. The summary of the criteria and their components are illustrated in Figure 1. Details regarding the components and their numerical ratings will be explained.

Figure 1:Summary of the proposed criteria eHealth Ontario should use to evaluate health apps and their associated components.

Scientific Relevance

Scientific relevance is a key criterion for the testing of any health app. The first component for its evaluation is its evidence-based principles. Core functions of the app should be supported with scientific literature and knowledge⁵. If an app proposes changes in patient actions, it should do so in accordance with the values of the Canadian healthcare system. When rating this component, a score of 0 means none of the core functions of the app are supported with evidence. A score of 4 is given when all core functions are completely supported with evidence. Scores of 1 to 3 are given accordingly based on the varying degrees of evidence.

Information quality is the second component of its evaluation. Consumer health apps require high levels of information quality ⁶. It refers to the accuracy of information within the app such as suggestions and facts. It also includes the ability of the app to collect accurate information about a patient, whether through manual or automatic means (such as through an external device e.g. fitness tracker). A score of 0 is given when all the information is of poor quality, while a score of 4 is given when all the information is of high quality.

Behavioural Effect

Positive behavioural effect is another key determinant in the success of a health app. The ability of an app to beneficially affect the behaviour of a user is necessary for health improvement, hence, its overall function should be in line with behavioural science³. The Self-Determination Theory has been used in this report to evaluate behavioral effect through its components of autonomy, competence, and relatedness. Satisfaction of these components leads to more health conductive behaviors and improved mental health⁷.

Autonomy represents the ability of the app to influence user behaviours based on individual values and interests. The app should operate and plan according to user perspectives in order to promote engagement³. Competence is the ability to promote confidence in changing behavior. Regular feedback can help facilitate competence³. Relatedness refers to apps influence on patient’s connectedness with others, allowing them to feel understood and cared by others^3,7. For each of these components, a score of 0 means the app made no successful attempts at addressing them, while a score of 4 means all attempts were successful.

Healthcare Response

Healthcare response is a criterion that outlines an apps ability to integrate with external health services. The component ‘treatment options’ represents the need of connecting patients to available clinical settings. Based on the function of the app, treatment options should be presented with geographic relevance⁸. Not only does this facilitate the interoperability of healthcare, it also promotes the Health Canada need for expanding health outreach⁹. Counseling is a component that refers to the ability to advise patients based on their progress. It should be accurate and relevant to the patient. For each component, scores of 0 mean that no treatment options are provided, and no counseling is provided. Scores of 4 mean that treatment options and counseling are provided in an accurate, relevant and timely fashion.

Usability

Usability is an important criterion for the evaluation of an app as it represents the quality of its contact with the patient. It is described using the components of interaction design and accessibility. Interaction design is evaluated using an altered and condensed version of Jakob Nielson’s ten general principles (Table 1)¹⁰. Each of the four principles is given a score of 0 if it is not satisfied, and 1 if satisfied, with a maximum total score of 4.

Table 1: Description of the principles used to evaluate the interaction design of apps. These principles were derived from Jakob Nielson’s ten general principles of interaction design¹⁰.

Principle	Description
User Literacy	App should utilize words and concepts that are familiar to the user. Terminology to be matched with the general education level of patients11.
Intuitiveness	The app should be intuitive, allowing users to function without prior knowledge. App should also be easily controlled with interactions such as exit, save, edit and go back.
Aesthetic Design and Efficiency	Irrelevant information is excluded, providing a simplified experience for the user.  Users can also accomplish tasks quickly and efficiently.
Help and Documentation	Further information regarding app function and explanation for errors are provided.

Accessibility is a component of usability that accounts for users with various disabilities. Options should be available for vision requirements older populations like larger sizing of elements and color contrast. Attempts should also be made for further vision problems with features like voice over and talk back¹¹. In general, it should only be required to provide features for limited vision.  Full score in this component is when all accessibility features for vision are included and are effective.

Patient-Centeredness

Patient-centeredness is a key criterion that ensures an app meets the desires of patients. User needs is truly a significant component in evaluating this criterion. While an app cannot address all user needs of a target population, it should attempt to address some of them to ensure it is relevant. Full score in this component can be achieved when the core function of the app successfully addresses patients needs.

Privacy and security is a necessary component of evaluation due to the patient and governmental need to protect patient information. Various features need to be included to ensure that the privacy and security of users are maintained². These features are: password login, encryption of information, and appropriate data destruction methods. Data destruction refers to the complete deletion of user information after app and account deletion. Complete and appropriate satisfaction of these privacy and security requirements will yield a score of 4.

Each criterion is equally valued at 20% of the total consumer health app evaluation. The components of each criterion will have their individual scores out of 4 totaled, and the fractions will represent 20% of the total of evaluation. The sum percentage of all criteria will represent the overall rating of the app. eHealth Ontario should not endorse an app that scores below 75%.

Withings – Health Mate

Withings is a company based in France, that focuses on creating consumer health electronics. Weight scales, blood pressure monitors, and activity trackers are some of their most popular products. On April 26^th 2016, Withings was purchased by Nokia, facilitating their effort to expand into the market for health monitoring¹². Among their many product developments, ‘Health Mate’ is an app that aims to integrate data from their various products, as well as provide other aspects of health management (it can also be used as a standalone app without any extra hardware). Health Mate’s core function is to monitor weight, activity, sleep and heart rate.

Weight

Weight can be logged into the app manually, or automatically using a Withings Wi-Fi smart scale. Users can step on a scale as they normally would, and their weight would automatically be recorded on their app. In addition, body composition metrics like body fat and water percentages can be logged. After successfully inputting enough measurements into the app, the complete data history is illustrated on a graph and can be manipulated by date (Figure 2). Initially, users are prompted to enter their desired weight and the timeframe in which they want to achieve this goal. Logged data can be viewed relative to their desired goal, and the app occasionally offers insight regarding progress.

Activity

Activity can also be logged manually or automatically using various activity trackers. When inputted manually, users can specify the activity type, intensity, start time and duration. The app automatically calculates how many calories were burned based on height and weight data. Steps can be counted automatically using a smartphone’s built-in accelerometer. As is the case with weight, users can view their history using informative graphs, as well as compare it with specified goals.

Sleep

Unfortunately, sleep can only be tracked automatically using various devices. Health Mate can log sleep cycle variation and duration. It also offers insight regarding the factors that may affect users sleep quality. Once again, all information is illustrated on detailed graphs.

Heart Rate

While heart rate cannot be manually logged, it can be measured without using an extra device by placing a finger on the smartphone camera. External devices such as activity trackers can also be used to log this data. Heart rate data can be viewed on graphs and compared to normal value ranges. If heart rate is abnormal and not in a stressed situation, users will be alerted to seek medical attention.

Motivation

Health Mate has many features to motivate their users. Leaderboards allow friends and family to compete and compare activity goals. This feature is only used with step measurements. Badges and virtual rewards are also used for motivation. Users can earn badges for reaching a goal, however, this feature is also only available with activity tracking.

Reminders

Health Mate has a system in place for users to activate various reminders. This function is not limited to reminding users to log health data. It can be used to alert users to engage is various beneficial health practices (Figure 3). For example, in an attempt to maintain a healthy weight, users can be reminded not to skip breakfast. To supplement daily activity, users can be reminded to walk while making phone calls. Heart function can be improved as users can be reminded to control salt intake.

It should be noted that Withings offers true interoperability features regarding its data. Health data can be easily exchanged between apps like MyFitnessPal and Apple Health. Data can also be easily exported in various formats, while also including the option to send the data directly to a physician’s email.

                               

Health Mate Analysis

The previously mentioned 5 criteria, and their respective components were used to evaluate Health Mate. Successful evaluation, a score of at least 75%, would mean that Health Mate should be eligible for endorsement by eHealth Ontario. The results of the evaluation are summarized in Table 2.

Table 2: Evaluation of Health Mate to determine eligibility for endorsement by eHealth Ontario. The reasoning for assigned scores are provided.

Criteria	Component	Score	Reasoning
Scientific Relevance	Evidence Based Principle	4	All core functions supported in the literature. Promoting daily tracking of weight shown to beneficially affect weight loss13. In general, promoting users to seek numerical knowledge about themselves promote reaching health targets14. Overall reminders and feedback regarding individual data points (normal BMI range, normal heart rate range) correlate with literature. Benefits of self-tracking in line with the Health Canada objective to promote healthier lifestyles9.
	Information Quality	4	All app information is of high quality. The method used to manually enter data facilitates accuracy through ease of use, clear function and ability to edit. Automatic data entry from external devices is as accurate as the devices themselves.
Behavioral Effect	Autonomy	4	App allows for complete autonomy. User can set goals based on their needs and preferences. Feedback and preferences revolve around these goals. User can choose to use only specific features in the app
	Competence	1	Poor ability to provide user competence in a complete and extensive manner. While aspects such as virtual badges may provide confidence, it is a feature only found in activity tracking. Feedback is presented in a way that does not significantly alter behaviour. It only shows that a user is out of normal range. Feedback may also be given as a numerical representation of reaching one goal (e.g. 2 pounds to go to reach monthly target)
	Relatedness	3	Average ability to encourage relatedness in a complete and extensive manner. Attempt at gamification through the use of leaderboards is only available with activity tracking and not weight, heart rate or sleep. Great ability to share all data to whoever user desires.
Criteria	Component	Score	Reasoning
Healthcare Response	Treatment Options	2	Average ability to suggest relevant treatment options to improve health. The app will suggest seeing a physician if extreme values are recorded. However, no detail regarding what kind of physician or any local Ontario options are provided. Some useful reminders for improving health outcomes like water intake, salt intake, and caffeine intake. However, those are simply scheduled reminders and do not reflect individual health performance.
	Counseling	1	Poor and limited ability to provide relevant counseling. Counseling only revolves around letting the user know if logged data falls within a healthy range.
Usability	Interaction Design	4	Complete satisfaction of all interaction design principles (Table 1) User Literacy – Easy to understand concepts and terminology. Intuitiveness – Easy to grasp interface. Maneuvering between pages performed easily. Aesthetic Design and Efficiency – No Irrelevant information. Very simple and to the point. Tasks can be performed quickly with little downtime. Help and Documentation – Complete documentation covering all aspects of app function.
	Accessibility		No attempts to include features that support potential sub-optimal vision of users. Accessibility features found in iPhone iOS (larger text and voice-over) do not carry over to Health Mate
Patient Centeredness	User Needs	4	Core functions of the app successfully meet the needs of users. Major user needs for patients, especially with chronic disease, revolve around the ability to self-manage their disease. This can include tracking heart rate, weight, and physical activity15,16.
	Privacy and Security	2	Average satisfaction of privacy and security requirements. No detailed information is provided regarding data deletion or security measures. Mandatory account passwords. The privacy policy states that data is stored on servers located in France and secured using advanced techniques and procedures17. No detail regarding what encryption techniques are provided. The privacy policy states that while data may be deleted when the account is deleted, it still may remain on Withings security backup servers17. No information available regarding how data is destructed, and if it gets deleted from backup servers. Poor transparency for these details yielded the average score of 2.

Table 3: Calculation of overall evaluation rating of Health Mate.

Criteria	Component	Score	Maximum Possible Score	Combined Criteria Score Out of 20%
Scientific Relevance	Evidence Based Principle	4	4	20
	Information Quality	4	4
Behavioral Effect	Autonomy	4	4	13.34
	Competence	1	4
	Relatedness	3	4
Healthcare Response	Treatment Options	2	4	7.5
	Counseling	1	4
Usability	Interaction Design	4	4	10
	Accessibility		4
Patient Centeredness	User Needs	4	4	15
	Privacy and Security	2	4
TOTAL				65.84%

With a score of approximately 66%, Health Mate falls short of the 75% threshold needed to be endorsed by eHealth Ontario. Key areas of concern involve the criteria of healthcare response and usability. Health Mate needs to do a better job of integrating available treatment options available in Ontario. More extensive and personable counseling options should also be provided. While the app had a high interaction design score, it failed to include any accessibility features. By attempting to modify these two criteria, Health Mate will pass this test of endorsement.

References

1. Smartphone behaviour in Canada and the implications for marketers in 2016. Catalyst, 2016. (Accessed February 5, 2017, at http://catalyst.ca/2016-canadian-smartphone-behaviour/.)

2. Atienza AA, Patrick K. Mobile Health. American Journal of Preventive Medicine 2011;40:151-3.

3. Mendiola FM, Kalnicki M, Lindenauer S. Valuable Features in Mobile Health Apps for Patients and Consumers: Content Analysis of Apps and User Ratings. JMIR mHealth uHealth 2015;3:40.

4. Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Translational Behavioral Medicine 2014;4:363-71.

5. Hasman L. An Introduction to Consumer Health Apps for the iPhone. Journal of Consumer Health On the Internet 2011;15:322-9.

6. Stoyanov RS, Hides L, Kavanagh JD, Zelenko O, Tjondronegoro D, Mani M. Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps. JMIR mHealth uHealth 2015;3:27.

7. Ng JYY, Ntoumanis N, Thøgersen-Ntoumani C, et al. Self-Determination Theory Applied to Health Contexts. Perspectives on Psychological Science 2012;7:325-40.

8. Abroms LC, Padmanabhan N, Thaweethai L, Phillips T. iPhone Apps for Smoking Cessation: A Content Analysis. American Journal of Preventive Medicine 2011;40:279-85.

9. About Mission, Values, Activities. Health Canada, 2016. (Accessed February 6, 2017, at http://www.hc-sc.gc.ca/ahc-asc/activit/about-apropos/index-eng.php#obj.)

10. 10 Usability Heuristics for User Interface Design. Nielsen Norman Group, 1995. (Accessed February 6, 2017, at https://www.nngroup.com/articles/ten-usability-heuristics/.)

11. Arnhold M, Quade M, Kirch W. Mobile Applications for Diabetics: A Systematic Review and Expert-Based Usability Evaluation Considering the Special Requirements of Diabetes Patients Age 50 Years or Older. J Med Internet Res 2014;16:104.

12. Nokia to buy health tracking firm Withings. Reuters, 2016. (Accessed February 6, 2017, at http://www.reuters.com/article/us-withings-m-a-nokia-corp-idUSKCN0XN0MA.)

13. Pacanowski CR, Bertz F, Levitsky DA. Daily Self-Weighing to Control Body Weight in Adults. SAGE Open 2014;4:16

14. Kim J. A Qualitative Analysis of User Experiences With a Self-Tracker for Activity, Sleep, and Diet. Interactive Journal of Medical Research 2014;3:e8.

15. Agarwal R, Light RP. Sleep and Activity in Chronic Kidney Disease: A Longitudinal Study. Clinical Journal of the American Society of Nephrology 2011;6:1258-65.

16. Chiauzzi E, Rodarte C, DasMahapatra P. Patient-centered activity monitoring in the self-management of chronic health conditions. BMC Medicine 2015;13:77.

17. Privacy Policy. Withings, 2015. (Accessed February 6, 2017, at https://www.withings.com/fr/en/legal/privacy#/fr/en/legal/privacy.)