Discovery AI project aimed at increasing user engagement in a double-sided job marketplace

Stepstone | Enterprise | Responsive web | B2C

Summary

I was a Product Design Lead at Stepstone, a double-sided job marketplace. I led a 5-month discovery project resulting in a responsive web AI-powered interview coach, enabling job seekers to practice mock questions and get actionable feedback on performance.

Goal

Increase user engagement and extend our service beyond traditional job applications, to provide a better job seeker experience.

Impact

12% increase in user engagement with 78% of users stating they felt more prepared for their interview after using the tool.

It was also presented at the Microsoft AI Conference (Envision).

Responsibilities

  • Design and Stakeholder Management

  • Research (Qualitative and Quantitative)

  • Design Sprint Facilitation

  • Product Design

  • UX Writing

  • Prototyping

ORGANIZATION AND MY ROLE

Stepstone is a double-sided job marketplace

Their core service connects job seekers and companies across 30 markets. I joined the organization to lead their discovery practices in an engagement portfolio, with an emphasis on better supporting our users. The platform processes 9.3M job applications per year.

I focused on discovery projects, coaching others and improving ways of working.

THE PROBLEM AND GOAL

Job seekers become inactive after submitting applications, not returning to the platform for another 4 years

We needed to improve user engagement to increase customer lifetime value

THE WORK BEFORE THE WORK

To ensure alignment I interviewed leadership stakeholders, created a Slack channel and facilitated a project kick-off

It was important to understand the goals and concerns of leadership stakeholders before jumping in to group conversations, so we could be aligned from the start.

DECIDING WHERE TO FOCUS

To set our direction I analysed a Jobs-to-be-Done study, and ran a survey with a product manager and a researcher

We examined the importance vs satisfaction of user needs throughout the journey of a job seeker, analyzing all stages from deciding to look for a new role through to securing one. Aligning these needs with our platform's existing experience, we highlighted three areas with significant potential for impact. I facilitated a session with leadership to decide which area we wanted to focus on.

We decided to concentrate on improving the experience of interviewing, as it aligned with our long term business objectives and is the logical next step for job seekers post-application.

DIVING IN TO THE NEEDS OF USERS

To understand the needs of job seekers and hiring managers experiences when interviewing, I collaborated with a researcher

Activities

  1. Interviewed 15 job seekers to understand their pain points, needs, and experiences with interviewing.

  2. Interviewed 6 hiring managers from different industries to understand their goals in interviews.

  3. Surveyed 1,000 job seekers to understand how and when they prepare for interviews.

  4. Conducted a competitive study with a business analyst to analyze 9 interviewing support tools.

Key insights

  1. Job seekers struggle with foreseeing interview questions, formatting answers, and getting actionable feedback on how to improve.

  2. Job seekers want to understand their mistakes, make a good first impression, and enhance their overall interview performance.

  3. Job seekers often practice for interviews at home with friends and family, but they may perceive this feedback as biased, while online tools and resources offer limited support with generic tips.

  4. Recruiters want to get a better picture of a candidate, understand if they align with the company culturally and expect clear, concise responses.

PEOPLE WHO CAN BENEFIT FROM SUPPORT WITH INTERVIEWING

GETTING STARTED WITH IDEATION

To kick start our ideation, I co-facilitated a design sprint with a product manager

We invited 10 people along, including directors, researchers, product designers, Large Language Model (LLM) engineers and a recruiter. It was important to include a diverse range of perspectives as interviewing is a broad and complex topic. In the sprint we played back the research, agreed on a long term goal, generated ideas, created solution sketches, agreed on a concept, fleshed out a storyboard, and created an interactive mid-fidelity prototype.

It was a long week which helped us explore different directions quickly.

THE DESIGN SPRINT CONCEPT

Collaborating with two designers we created an AI powered mock interview coach concept, which provided job seekers with actionable feedback on their performance

We tested the concept in 6 moderated interviews to gather insights.

HOW IT EVOLVED OVER TIME

To evolve the project post-design sprint, we crafted and tested 4 different iterations

EXPLORING OTHER DIRECTIONS

To understand the capabilities of body tracking technology, I worked with a team of engineers to build a proof of concept

During the design sprint we’d generated a concept which used body tracking technology, but at the time we couldn't delve into it due to prototyping limitations and time constraints. After the sprint, I collaborated with engineers to investigate the technology further, creating a proof of concept that analyzed facial and body movements to predict a users emotions. The concept would provide users feedback on how to be more comfortable when interviewing.

It was cool, but we found the technology would be expensive and overly complex for our project so decided to park any further discovery.

VISION DESIGN

I designed a responsive web experience which allows job seekers to practice personalized interview questions using speech-to-text technology, providing them with objective feedback for improvement

I collaborated with our design systems and brand team to create new UX patterns, Large Language Model (LLM) specialists to refine prompts, and engineers to slice and deliver and iterate on the product.

IMPACT AND SUCCESS

ENGAGEMENT

Increased user engagement by 12%

USER SUCESS

78% of users felt more interview ready after using the tool