Mental health is a crucial aspect that should never be overlooked. Its impact reaches far beyond just the individual, affecting society's productivity and the global economy. Sadly, 1 billion people or ~13% of the world's population suffer from mental disorders, with two of the most prevalent being anxiety and depression. The economic impact of anxiety and depression, alone, results in a yearly loss of US $ 1 trillion globally in terms of lost productivity. Investing in mental health treatment for depression and anxiety offers a strong economic return, with a $4 improvement in health and productivity for every $1 invested in scaled-up treatment.
Unfortunately, the stigma surrounding mental health can make it difficult for people to seek help, and even when help is desired, it may not be easily accessible. The advent of smartphones has made mental health apps a useful tool in addressing conditions such as anxiety and depression. The use of these apps is cost-efficient and can be easily expanded. The market for mobile mental health apps is growing rapidly, with global expenditures on these apps increasing from $200 million in 2019 to an estimated $500 million in 2022. There are currently around 20,000 mental health apps available.
To gain a better understanding of user experiences, I decided to investigate one of the top-rated mental health apps, Talkspace, by analyzing customer reviews.
The first step in the analysis was acquiring the data. The focus was on reviews from the Google Play Store. The reviews on the Google Play Store website are dynamically loaded and to see all of them, one has to scroll to the bottom of the page and click on "See all reviews." This leads to a pop-up window whose HTML code updates as you scroll through the reviews. Hence, both Selenium and BeautifulSoup were used to scrape the data. The Selenium package in python allows one to conveniently connect with Webdriver protocol to control web browsers like Chrome, Firefox, or Safari. Once the full html code was captured using Selenium, I used BeautifulSoup to extract the review content, date of review, name of reviewer, and the number of stars given to the review. Here is the code I used:
Import the required packages
from selenium import webdriver
from selenium.common.exceptions import StaleElementReferenceException,
TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import pandas as pd To instruct the webdriver to continue scrolling until the html elements that reference reviews on the webpage are no longer present, a callable class was first created. The code provided below was sourced from here. The object created by the
wait_for_more_than_n_elements_to_be_present class is made callable using the __call__ method. An instance of this class has two attributes: (i) locator and (ii) count. In this case the locator was the Xpath of the review body in the html code. The count was the number of reviews.
class wait_for_more_than_n_elements_to_be_present(object):
def __init__(self, locator, count):
self.locator = locator
self.count = count
def __call__(self,driver):
try:
elements = EC._find_elements(driver, self.locator)
return len(elements) > self.count
except StaleElementReferenceException:
return False The reutrn_html_code function includes a while loop that invokes the class. The process begins by locating the web element that references the review data (date, content, reviewer name, star rating), and recording the initial count of reviews. The webdriver scrolls down to the last review, and then pauses for additional reviews to load. This continues until the count of relevant web elements currently present is no longer greater than the count of those present before the scroll.
def return_html_code(url):
options=webdriver.ChromeOptions()
options.add_argument('your user agent')
options.add_argument('--disable-blink-features=AutomationControlled')
options.add_experimental_option("excludeSwitches", ["enable-automation"])
driver = webdriver.Chrome(executable_path='chromedriver',options=options)
driver.get(url)
##allow See all Reviews button to appear then click it
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[@class='VfPpkd-LgbsSe VfPpkd-LgbsSe-OWXEXe-dgl2Hf ksBjEc lKxP2d LQeN7 aLey0c']"))).click()
##allow pop-up window showing all reviews to appear and reviews to load
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[@class='RHo1pe']")))
##keep scrolling until no more elements of interest are present
while True:
reviews = driver.find_elements_by_xpath("//div[@class ='RHo1pe']")
number_of_reviews = len(reviews)
driver.execute_script("arguments[0].scrollIntoView();", reviews[-1])
try:
wait.until(wait_for_more_than_n_elements_to_be_present((By.XPATH,
"//div[@class ='RHo1pe']"), number_of_reviews))
except TimeoutException:
break
html_full_source=driver.page_source
driver.close()
return html_full_sourceAfter extracting the relevant web elements, Beautiful Soup can be used to parse the HTML source code and obtain the necessary information.
#access html source code on google play talkspace
html_source=return_html_code('https://play.google.com/store/apps/details?id=com.talkspace.talkspaceapp')
#Load html using Beautiful soup
soup=BeautifulSoup(html_source,'lxml')
#Extract reviewer name, date of review, review text, and number of stars given by the reviewer
review_dict = {'name':[], 'date':[], 'review':[], 'rating':[]}
for review in soup.find_all('div', class_='RHo1pe'):
review_dict['name'].append(review.find('div', class_='X5PpBb').text)
review_dict['date'].append(review.find('span', class_='bp9Aid').text
review_dict['review'].append(review.find('div', class_='h3YV2d').text)
review_dict['rating'].append(review.find('div', class_='iXRFPc')['aria-
label']) Out of the 5,780 reviews available on the Google Play store, only 2,442 were successfully extracted. Despite attempting to use the Google Play store scraper, similar results were obtained. Despite this, I chose to proceed with the limited data, as my aim was to gain insight and not to construct a sentiment prediction model.
For the analysis, only English reviews were considered. Reviews in Russian, Spanish, and Vietnamese were removed from the dataset. Only reviews from the period of 2020 to 2022 were analyzed, and reviews with a neutral rating of 3 stars were not included, nor were reviews that consisted of a single word. The average rating in the analyzed data was 1 star, with roughly 23% of the reviews being high (4-5 stars) and 77% being low (1-2 stars).
Once, the dataset was finalized, the next step was to preprocess the text. This included:
normalizing case by lowercasing all text
removing punctuation
lemmatizing text (normalize words to their base root mode)
removing stop words (commonly occurring words that don't carry significant meaning)
The removal of stopwords was a two-step process. Initially, the built-in stopword list from the Python NLTK package was applied. Then, the inverse document frequency (IDF) of the remaining terms was calculated to eliminate domain-specific terms. IDF quantifies how rare a term is, with rarer terms having a higher IDF. Words with an IDF in the bottom 15th percentile were removed, with this cutoff being selected arbitrarily. Additionally, adjectives such as "awesome," "excellent," "awful," and "useless" were eliminated to focus on specific features and experiences that users liked or disliked about the app.
The following step involved visualizing the text. One common method is creating a word cloud, where the size of a word is determined by its frequency of occurrence. However, this approach can obscure important words that do not appear frequently. To address this, I utilized the term frequency inverse document frequency (TF-IDF) of each word. This method takes into account not just the frequency of a word within a document (in this case a review), but also its rarity in the overall corpus (all reviews). I generated word clouds using both raw word counts and the TF-IDF values of each term. I also created a wordcloud that displayed the most frequently occurring bi-grams (pairs of consecutive words/tokens in a text).
While many users appreciate the concept of having mental health assistance easily accessible through their phones, not all users believe that Talkspace effectively executes this idea. The analysis of user reviews through wordclouds generated, from both raw word counts and TF-IDF, uncovers common words such as agent, confirm, multiple, and spend. However, the relative importance of these words differ between the two methods. Additionally, the TF-IDF wordcloud surfaces other words such as phone, log, register, slow, and nickname that reveal key concerns and technical issues.
Talkspace doesn’t provide a telephone support service for customer assistance, relying instead on a ticket system that is not well-suited to urgent issues. Users report difficulties in connecting with matching agents, who are often unresponsive or inattentive. The app's scheduling functionality is also criticized as being inadequate, making it difficult for users to confirm appointments. Technical issues, such as frequent logouts, difficulty logging in or registering, and problems with setting passwords and nicknames are also commonly reported. Users also express concern about being required to provide credit card information in order to register and about a lack of transparency in how this information is used. Users report being charged even when they have cancelled their subscriptions or while waiting for therapists to reschedule an appointment. Therapists are also reported as being slow to respond, which may be due to being overbooked. The app is also prone to not loading, making it difficult for users to see messages from therapists. Overall, users have struggled with a range of issues while using Talkspace, making it difficult for them to access the mental health assistance they need.
Taking a look at the wordclouds generated for the high rated reviews, complaint, safe, accessible, and tech are words that are highlighted in the wordcloud generated using TF-IDF. Users greatly appreciate the voice memo feature and the ability to express their emotions in real-time through writing, making therapy more convenient and accessible. Talkspace has been particularly helpful for those with social anxiety. However, some users from this group also feel that the overall process could be made more user-friendly. Users expressed their desire for additional features such as an (i) an option to spread out payments (ii) the ability to hide therapists you no longer see (iii) ability to pause /rewind/fast forward recorded messages.
Talkspace has made a significant impact in the lives of many, offering support to those with social anxiety through text messaging and voice memos and aiding those with depression and other anxiety disorders. The user experience can be enhanced by resolving the bugs or technical issues during the registration process, managing subscriptions effectively, allowing for easy provider changes, and ensuring efficient appointment confirmation processes. In addition a telephone support system could help users resolve any issues they encounter more efficiently. To increase transparency, Talkspace can create an informational guide that outlines all of its features, available plans and their costs, therapists and their credentials, billing practices, as well as their privacy policy and regulatory practices. To enhance response times, Talkspace can maybe first assess the ideal therapist-to-client ratio and then gradually increase both the number of users and therapists. It's crucial that users relying on a mobile app for their mental health have access to real-time support. By addressing the above issues, Talkspace can continue to make a positive impact on the lives of those who rely on it.
References:
1. The Lancet, “Mental health matters,” Lancet Global Health 8, no. 11 (2020)
2. Auxier, B., Westcott, K., & Bucaille, A. (2021, December 1). Mental Health goes mobile: The mental health app market will keep on growing. Deloitte. Retrieved December 31, 2022
Commenti