50% Increase In User Retention with Noom
Testimonial
"Another key skill in Akua is her ability to ask the right questions- it helps her develop better plans and solutions that really solve the problem at hand. "- Mallori Hensley, Regional Director Noom Inc.
Outcome
The What: Problem
Noom, a behavior change start up app, was facing challenges making organizational changes like adding new tools and systems, which lead to a decrease in coach satisfaction and decline in impact on user retention/satisfaction
As a fast-growing startup, Noom began implementing company changes early 2020 (performance management systems, new coaching tools, etc) to aid in creating organizational structure. Stakeholders wanted this added structure to enhance coach impact, possibly increasing user retention rates and satisfaction scores. Noom used its research-based approach to support not only users but also employees, sending out monthly and quarterly employee feedback surveys to survey employee sentiment. The surveys showed that:
over a 6-month period, more than half of the coaches had a negative sentiment toward the company changes.
The current implementation strategy was:
influencing coach burnout and dissatisfaction
leading to a decrease in coaching impact correlated with reduced user retention rates and satisfaction scores
Coaching management wanted to figure out the "why" behind this pattern and determine the "how" to improve it.
The How: Process & Methods
Drove Coach Satisfaction and Metrics Improvement through Applying Feedback from Focus Groups
As part of the user experience coaching management team, we consistently reviewed metrics such as customer satisfaction scores, customer retention rates, and collected feedback to assess the effects of new features and coaching protocols. Through this review, I saw a negative correlation between coach satisfaction and customer retention, thus led the initiative within my department to improve these metrics.
My role involved leading, coordinating, and facilitating research from start to finish by:
Objective:
Verify coaches sentiment towards new company changes and coaching model to better improve coach sentiment and positively impact coach metrics (customer satisfaction and retention).
Methodology:
Step 1 - Stakeholder Meetings: Collaborating with Coach Management Teams
During this step, I worked closely with the coach management team to review the survey analysis and compile a list of topics that coaches expressed interest in exploring further. Once we had gathered these topics through team specific slack surveys, we considered the most effective methods for deeper exploration. Knowing that surveys were used to provide quantifiable data across the company and we were aiming to dig deeper into the 'why' and 'how' behind company sentiments.
We chose focus groups to:
create an environment where coaches could openly share their experiences, so they didn't feel isolated in their sentiments
quickly gather more insights from a large and diverse group of people
Keeping in mind that:
group conversations might silence or influence coaches who had a different experience than the group majority
Step 2 - Creating a Guideline for Discussions: Setting Clear Goals
The themes identified in the company and team wide survey results pin pointed two key issues:
poor communication between the organization and the coaching team
poor collaboration between coaches with each other and leaders
I created a discussion guideline to organize the flow of conversation and set clear goals for the focus groups. We aimed to use the discussion groups to explore these sentiments and discuss ways to improve it.
Step 3 - Facilitating and Moderating Remote Focus Groups: Collecting Feedback
I partnered with managers to facilitate and moderate:
two remote focus groups
34 participants total
I also incorporated virtual surveying tools such as Mentimeter to allow for anonymous feedback in the group space.
Step 4 - Summarizing Key Takeaways and Presenting Insights
I summarized the insights from the focus groups and presented them to senior management. There was reoccurring feedback to create some sort of collaboration and open communication panel which was the resulting recommendation.
The What Else: Learnings
Enhanced Coaching Sentiment by 82% and User Retention by 50% through Improved Coach Collaboration and Leadership Transparency
Results:
The feedback collected from coaches reinforced what we knew:
The lack of communication and collaboration was having a negative impact on coach wellbeing and coach-client relationships
We learned that:
Coaches believed more collaboration and transparency had the high chance of improving coaching sentiment
76% of coaches expressed positive feelings toward creating methods for collaboration within their daily work expectations
Next Steps:
We implemented quarterly collaboration panel for coaches, where coaches from across our department could get together virtually. This space would be used to share ideas, tools, and resources to enhance the coaching experience. The first collaboration panel was held shortly after the end of the research project. This allowed coaches the space to immediately implement their feedback.
After the following quarter:
Lessons:
Reducing the size of focus groups
Having 15 coaches participate in a 30 minute session and going over time in both sessions, I quickly learned that having too many participants can:
reduce the quality of feedback
cause participants to feel rushed or overpowered
Even with breakout rooms. Thankfully, people were engaged and willing to discuss longer. In the future, recruiting 5-8 participants per focus group will allow more time for open discussion and clarifying insights.
Virtual Focus Groups
Pros
Easier to connect with more people
Flexible for research
Cons
Connection issues: Preparing in advance to make sure all systems are working
Lacks open forum feel: We used breakout rooms to help balance this out. Having a chat moderator could also have created balance.