Overview
Experimentation drives innovation, but inefficient tools were slowing our users—engineers, data scientists, and product managers—hindering their ability to test, learn, and iterate.
To address this, I led Product Discovery, collaborating with engineering and data science to analyze usage data, gather insights, and review past feedback. This deep dive into user experience uncovered major challenges and directly shaped product strategy—ensuring insights informed key decisions and the roadmap.
Understanding the users
The platform serves a diverse set of users, each playing a critical role in the experiment-driven rollout lifecycle:
- Implementation: Engineering builds new features and sets up tracking for experimentation.
- Experimentation: Data Science tests hypotheses, analyzes results, and uncovers insights.
- Rollout: Product Management coordinates feature rollouts, ensuring a smooth transition from testing to production.
Once features are fully rolled out, some experiment variables remain adjustable, while others become embedded in the code. Understanding how users navigate this lifecycle was essential to designing solutions that truly fit their needs.
Leading Product Discovery to Shape Strategy and Direction
Through Product Discovery, I worked cross-functionally to analyze usage data, gather insights from internal users, and review past feedback. This research surfaced two major issues:
- The Experimentation Dashboard, needed optimization to improve speed and discoverability for a better user experience.
- The Experiment Creation Flow required improvements to simplify the process, making it more intuitive and accessible for new users.
To address these challenges, we focused on streamlining the experience—enhancing usability, improving discoverability, and creating a more intuitive, efficient platform.
Transforming the Experimentation Dashboard to Foster Collaboration and Ease Discovery
For years, the experimentation dashboard served as the heart of the platform, but it had become a source of frustration for users. The experience was slow, unintuitive, and no longer met their needs. Key issues included:
- Low Performance: Heavy, resource-intensive charts slowed down the page, making every interaction feel sluggish.
- Hidden Information: Critical details like experiment ownership or status were buried or inaccessible, forcing users to spend extra time searching.
- Fragmented Actions: Filtering options were scattered across the page, making it difficult to refine searches efficiently.
Recognizing these challenges, we completely reimagined the dashboard, designing a solution that puts users first. Here's what we changed:
- Streamlined Performance: We replaced the resource-heavy charts with a lightweight, efficient table design, drastically improving page speed and responsiveness.
- Unified and Centralized Filters: Filters were consolidated into a single area, including new options for metadata like city and country, enabling faster and easier search refinement.
- Redesigned Layout: The layout was overhauled to be intuitive, user-friendly, and scalable for future innovation. A new design system improved visibility, making important information like metadata clear and accessible at a glance.
Impact
- Faster Experiment Discovery: Experiment discovery speed significantly increased, enabling users to find relevant tests much faster.
- Improved Productivity and Satisfaction: By eliminating friction, we empowered users to focus on innovation and insights, rather than wasting time navigating the dashboard.
This transformation wasn't just about fixing problems, it was about creating a dashboard that drives efficiency, supports innovation, and empowers users to do their best work. What was once a source of frustration is now a tool that unlocks new possibilities.
Transforming the Experiment Creation Flow to Streamline Experimentation Velocity
Discoverability was a significant challenge, as previously mentioned, but the creation flow presented its own set of obstacles, particularly for new or beginner users. The journey of creating an experiment was fraught with complexity and confusion, stemming from several intertwined issues.
- Complexity and Cognitive Load: The existing creation flow was cumbersome and overwhelming. Users, especially those new to the platform, were left without clear guidance, often feeling lost and unsure of what steps to take next. This confusion led to skipped steps and incomplete setups, which hampered the overall experience.
- Layout Issues: The layout further compounded the problem. It was unclear whether users should progress from top to bottom, left to right, or alternate between sections. The absence of a logical visual hierarchy left the experience feeling disjointed and inconsistent, turning what should have been a straightforward process into a guessing game.
- Input Validation: To make matters worse, there was no real-time feedback to catch errors as users moved through the flow. Mistakes went unnoticed until much later, forcing users to backtrack, correct errors, and lose valuable time. Additionally, users had to rely on their memory to recall details they had entered in earlier steps, increasing the likelihood of mistakes and frustration.
Recognizing these pain points, we embarked on a journey to overhaul the experience, placing clarity and simplicity at the forefront. The solution was designed to empower users, streamline the flow, and reduce frustration at every step:
- Simplified Guided Flow: The experiment creation flow was reimagined to be more intuitive, reducing unnecessary steps and cognitive load. A guided experience with clear markers ensured accessibility for new users while retaining depth for power users. This streamlined approach extended beyond creation, maintaining consistency in viewing experiment setups and results.
- Real-Time Validation: We added real-time validation, a game-changer that eliminated the frustration of unnoticed errors. Now, users were immediately alerted to any mistakes, allowing them to fix issues on the spot without backtracking. This small yet impactful change significantly reduced error rates.
- Enhanced Cloning Experience: The cloning process was revamped to make it faster and more seamless, allowing users to build on past experiments with minimal effort and frustration.
Impact
- Error Rate Reduction: Error rates were reduced drastically through real-time validation, streamlining workflows and improving efficiency.
- Increased Experiment Velocity: Users now complete setups faster and with fewer mistakes, leading to faster innovation cycles.
Key Learnings
This experience deepened my understanding of designing for internal tools and the broader role of design in shaping workflows and company culture. Some key takeaways include:
- Building Continuous User Feedback Loops: Engaging with users shouldn’t be limited to problem-solving or feature launches. Establishing ongoing feedback loops—rather than reacting to issues as they arise—creates deeper visibility into user needs and behaviors. Regular touchpoints not only surface hidden pain points early but also foster stronger relationships with users, ensuring the product evolves in a meaningful way.
- How Platforms Shape Team Culture and Ways of Working: Platforms are more than just tools—they influence how teams collaborate, make decisions, and structure their work. Well-designed systems can drive efficiency, transparency, and alignment, while poor design can create friction and reinforce inefficiencies. Thoughtful design choices don’t just improve usability; they help shape how organizations operate at a fundamental level.
- Driving Adoption Through Thoughtful Change Management: A product’s success isn’t just about building the right features—it’s about ensuring people embrace and integrate them into their workflows. Even the best-designed solutions can face resistance if they disrupt established processes. Effective change management—through clear communication, structured onboarding, and ongoing support—is essential in driving adoption. Helping users understand the value of a new system and making the transition seamless ensures long-term success.
Designing for internal tools comes with unique opportunities and challenges, which I explore further in Beyond Interfaces: Designing Internal Tools.