50% Increase In User Retention with Noom

Testimonial

"Another key skill in Akua is her ability to ask the right questions- it helps her develop better plans and solutions that really solve the problem at hand. "- Mallori Hensley, Regional Director Noom Inc.

Outcome 

Impact of research study

The What: Problem

Noom, a behavior change start up app, was facing challenges making organizational changes like adding new tools and systems, which lead to a decrease in coach satisfaction and decline in impact on user retention/satisfaction

As a fast-growing startup, Noom began implementing company changes early 2020 (performance management systems, new coaching tools, etc) to aid in creating organizational structure. Stakeholders wanted this added structure to enhance coach impact, possibly increasing user retention rates and satisfaction scores. Noom used its research-based approach to support not only users but also employees, sending out monthly and quarterly employee feedback surveys to survey employee sentiment. The surveys showed that: 

The current implementation strategy was: 


Coaching management wanted to figure out the "why" behind this pattern and determine the "how" to improve it.

The How: Process & Methods

Drove Coach Satisfaction and Metrics Improvement through Applying Feedback from Focus Groups

As part of the user experience coaching management team, we consistently reviewed metrics such as customer satisfaction scores, customer retention rates, and collected feedback to assess the effects of new features and coaching protocols. Through this review, I saw a negative correlation between coach satisfaction and customer retention, thus led the initiative within my department to improve these metrics

My role involved leading, coordinating, and facilitating research from start to finish by: 

Research role responsibilities

Objective: 

Verify coaches sentiment towards new company changes and coaching model to better improve coach sentiment and positively impact coach metrics (customer satisfaction and retention). 

Methodology: 

Step 1 - Stakeholder Meetings: Collaborating with Coach Management Teams

During this step, I worked closely with the coach management team to review the survey analysis and compile a list of topics that coaches expressed interest in exploring further. Once we had gathered these topics through team specific slack surveys, we considered the most effective methods for deeper exploration. Knowing that surveys were used to provide quantifiable data across the company and we were aiming to dig deeper into the 'why' and 'how' behind company sentiments.

We chose focus groups to:

Keeping in mind that:

Step 2 - Creating a Guideline for Discussions: Setting Clear Goals

The themes identified in the company and team wide survey results pin pointed two key issues:

I created a discussion guideline to organize the flow of conversation and set clear goals for the focus groups. We aimed to use the discussion groups to explore these sentiments and discuss ways to improve it.

Step 3 - Facilitating and Moderating Remote Focus Groups: Collecting Feedback

I partnered with managers to facilitate and moderate:

I also incorporated virtual surveying tools such as Mentimeter to allow for anonymous feedback in the group space.

Mentimeter poll
Figure 1: Focus Group Moderation using mentimeter

Step 4 - Summarizing Key Takeaways and Presenting Insights

I summarized the insights from the focus groups and presented them to senior management. There was reoccurring feedback to create some sort of collaboration and open communication panel which was the resulting recommendation.

Coach feedback
Figure 2: Coach Feedback

The What Else: Learnings

Enhanced Coaching Sentiment by 82% and User Retention by 50% through Improved Coach Collaboration and Leadership Transparency

Results: 

The feedback collected from coaches reinforced what we knew:

We learned that:

Next Steps: 

We implemented quarterly collaboration panel for coaches, where coaches from across our department could get together virtually. This space would be used to share ideas, tools, and resources to enhance the coaching experience. The first collaboration panel was held shortly after the end of the research project. This allowed coaches the space to immediately implement their feedback.

After the following quarter:

Lessons: 

Reducing the size of focus groups

Having 15 coaches participate in a 30 minute session and going over time in both sessions, I quickly learned that having too many participants can: 

Even with breakout rooms. Thankfully, people were engaged and willing to discuss longer. In the future, recruiting 5-8 participants per focus group will allow more time for open discussion and clarifying insights.

Virtual Focus Groups 

Pros 

Cons