AI DEVELOPMENT PLATFORM
One Platform for the Entire LLM Development Lifecycle
COMPANY
Microsoft
ROLE
Product Designer
TOOL
Figma
TIMELINE
6 Months
Prompt Flow — Industrializing Prompt Engineering
From creative prompt experiments to enterprise- ready AI workflows
Prompt Flow is Microsoft’s internal platform that standardizes the end-to-end prompt engineering workflow.
It enables data scientists, AI developers, and product teams to collaboratively build, evaluate, and operationalize LLM-driven solutions with the same rigor as traditional software engineering.
My Delieverable
Role Coverage
Conducted research, including interviews, and marketing analysis.
Formulated strategic concepts, and defined project scope.
Crafting low to high-fidelity wireframes, and delivering the final UI design
Workflow Type
Authoring, Evaluation, Deployment
Impact
Faster iteration cycles, Traceable quality metrics, Scalable deployment
AI Engineers / Data Scientists / PMs
Product Overview
Create a New Prompt Flow: Select a pre-built template or blank page, Enhance the efficiency of prompt engineers.
Show Prompt Flow in Visual Way: DAG view reflect nodes in a flattened view.
Add and Set a Node: After adding an LLM node, users can set parameters, and edit prompt.
Link Nodes: Link each node by editing inputs, avoid confusion if there are multiple inputs in one node.
Test Prompt & Debug: Run flow to test prompt or use natural language directly in chat mode, to help prompt engineers understand the performance.
Run Specific Node to Check Performance
Run the Whole Flow to Check Performance
Collaborate with Team Members: In the workspace, check teammates' prompt flow, duplicate them as a reference, template, and help debug.
CHALLENGES OF TARGET USER - PROMPT ENGINEER
• High Learning Curve
Non-technical users often struggle with the complexity of AI tools, requiring more intuitive interfaces and simplified workflows.
• Lack of Collaboration
Developers and non-technical team members face communication barriers, resulting in delays and misaligned goals during AI integration.
• Limited Customization Options
Users need flexible tools to personalized AI outputs for diverse use cases, which are often restricted in existing products.
• Inefficient Iteration Process
Current workflows make it difficult to test and refine AI models quickly, hindering real-time innovation.
• Resource Constraints
Startups and smaller teams often lack the technical resources to fully leverage advanced AI functionalities.
HYPOTHESIS & GOALS
Providing an integrated, end-to-end platform for the entire LLM lifecycle, accessible to technical and non-technical users.
User Goals
Business Goals
Enable users to effectively design, test, and deploy AI workflows with ease, supporting non-technical users through an intuitive interface.
Position Prompt Flow as the leading LLM platform, driving Azure AI adoption and expanding its reach to diverse user groups.
PLANNING & PROJECT MILESTONE
Providing an integrated, end-to-end platform for the entire LLM lifecycle, enabling efficient development, deployment, and management of AI models.
Research
UNDERSTAND PROMPT ENGINEERING DOMAIN & INDUSTRY
Existing prompt engineering tools either prioritize simplicity for experimentation or advanced features for professionals, but fail to provide an integrated, user-friendly solution for the entire workflow.
Product Opportunity Gap
What we MUST HAVE
Existing tools either lack comprehensive functionality or are too complex, creating a need for a user-friendly, all-in-one platform for the LLM lifecycle.
A comprehensive platform with intuitive interfaces and advanced capabilities to support end-to-end LLM workflows effectively.
“I usually wonder if my prompts is good enough, and testing it in real-time can be challenging.”
#1 Simplified Prompt Validation
We started to play with 15+ prompt engineering products and summarized the insights from different aspects such as features, user flow, and UI pattern.
PAINPOINTS FROM INTERVIEW
What do our users struggle/concern with the most?
Uncertainty About Prompt Accuracy
Fragmented Workflow Across Multiple Tools
Onboarding Challenges for Non-Technical Users
KEY INSIGHTS FROM PAINPOINTS
DESIGN PRINCIPLE REVIEW
66% of participants expressed a lack of confidence in the accuracy of the prompts they create. Many noted the high cost and effort associated with repeated testing and refinement, leading to frustration and inefficiency.
80% of participants struggled with the need to switch between various tools for prompt writing, testing, refinement, deployment, and monitoring. This disjointed process made it difficult to maintain efficiency and focus, creating additional cognitive and operational burdens.
60% of participants highlighted difficulties in getting started with prompt engineering. Non-technical users often feel overwhelmed by the technical complexity and lack of guided workflows, frequently relying on external expertise or pre-existing templates to overcome initial barriers.
“I create prompts using CDEX, but deploying my chatbot with third-party products often takes significant time and effort.”
#2 Integrated Workflow for Efficiency
“I’m not a developer and I don’t actually know how to do with a blank sheet, so I ask help form my team’s expertise.”
#3 Accessible Onboarding for Non-Technical Users
We are focusing on providing transparent workflows and actionable insights throughout the prompt engineering process.
Empower users through intuitive control
Prompt Flow addresses user challenges by combining end-to-end integration with an intuitive interface that simplifies complex workflows. By unifying prompt creation, testing, and deployment in one platform, it minimizes inefficiencies and enhances productivity.
Define the MVP
DECISION MAKING METHODOLOGY
To align user goals with business goals while considering constraints,
We defined feature deliverables for the MVP stage using Product Roadmap.
Prioritization
MVP features
User Goals
We scoped features based on research insights and got feedback from designers, engineers, and product managers. My task is to design Prompt Flow development and evaluation.
Initialization
User guide
Workspace
Prompt Flow Template
Experimentation
Create & modify Prompt flow
Run flow against sample data
Evaluate prompt
Bulk Evaluation
Run flow with bulk dataset
Evaluate prompt
Production
Deploy & monitor flow
Expand Accessibility Across User Segments
Empower Non-Technical Users
Enhance Workflow Efficiency
Business Goals
Accelerate Adoption
The solution directly addresses core user needs
Essential features needed to launch MVP
Engineer resources & technical feasibility
💡How we decide on MVP features
Design Goals
Develop Intuitive, visually guided experiences for technical and non-technical users
Create an integrated workflow to eliminate the need for multiple tools
NARROW DOWN THE PROBLEM FOR THE MVP
How might we design a prompt evaluation system that provides clear metrics, simplifies the evaluation process, and supports technical and non-technical users to optimize workflow?
User Flow Iteration
After testing the initial user flow with developers, we discovered friction around the separate configuration steps for “Set Bulk Run” and “Set Evaluation.” To address this, I merged these two actions into a unified configuration step—Set Bulk Run & Evaluation Flow. This adjustment reduces one screen and step, resulting in a ~20% decrease in the workload related to bulk settings, while also simplifying the mental model for users.
Key Screens
AzureML Toolkit Based on Fluent 2 Web UI
I collaborated with fellow AzureML Design team members to build the AzureML Toolkit using the Fluent 2 Web UI framework.
Why it works
The toolkit enabled a consistent and cohesive design language across multiple AzureML functionalities, ensuring a unified user experience.
