AI Engineering
The AI Evaluation Playbook: Measuring What Matters in Production LLM Systems | AI PM Portfolio The AI Evaluation Playbook: Measuring What Matters in Production LLM Systems May 15, 2024 · 18 min read · Definitive Guide Most AI evaluation is done on benchmarks that do not reflect production reality. After 18 months of
AI Engineering
Multi-Modal AI in Practice: What Actually Works for Document Processing | AI PM Portfolio Multi-Modal AI in Practice: What Actually Works for Document Processing April 8, 2024 · 14 min read · Technical Deep Dive Multi-modal AI -- combining vision and text understanding -- promised to solve document processing in one shot. Send
AI Engineering
Building an AI Extraction Platform: 4 Providers, 60 Analyzers, 510 Tests | AI PM Portfolio Building an AI Extraction Platform: 4 Providers, 60 Analyzers, 510 Tests February 22, 2024 · 15 min read · Platform Architecture Case Study We built a document extraction platform that uses 4 AI providers, 60 specialized analyzers, and
AI Engineering
How We Cut AI Inference Costs 30x Without Sacrificing Quality | AI PM Portfolio How We Cut AI Inference Costs 30x Without Sacrificing Quality January 18, 2024 · 17 min read · Cost Optimization Deep Dive Our AI document processing pipeline cost $2.30 per document. At 128,000 documents per tax season,
AI Engineering
The AI PM's Year in Review: From Hype to Production in 2023 | AI PM Portfolio The AI PM's Year in Review: From Hype to Production in 2023 December 15, 2023 · 14 min read · Year in Review 2023 was the year AI went from demos to deployments.
User Research
25,000 AI Interactions Later: What Users Actually Want From AI | AI Product Manager Blog 25,000 AI Interactions Later: What Users Actually Want From AI By Dinesh · November 18, 2023 · 13 min read Last updated: November 2023 After analyzing 25,500 AI-assisted interactions at an AI-first tax platform, we
Startup
The YC Playbook Meets Regulated Industries: What Changes | AI Product Manager Blog The YC Playbook Meets Regulated Industries: What Changes By Dinesh · October 10, 2023 · 12 min read Last updated: October 2023 The standard YC advice is to move fast, talk to users, and iterate relentlessly. That advice is correct
Prompt Engineering
Prompt Engineering Is Product Design: A PM's Framework | AI Product Manager Blog Prompt Engineering Is Product Design: A PM's Framework By Dinesh · September 22, 2023 · 13 min read Last updated: September 2023 Prompt engineering is not an engineering task. It is product design. The prompt is
AI Engineering
15,000 Automated Assignments: Building Trust in AI Decision-Making | AI Product Manager Blog 15,000 Automated Assignments: Building Trust in AI Decision-Making By Dinesh · August 15, 2023 · 12 min read Last updated: August 2023 Users trust AI decisions more when they see the reasoning behind them, not just the outcome.
AI Engineering
Designing for 4.7/5 Satisfaction in an AI-First Product | AI Product Manager Blog Designing for 4.7/5 Satisfaction in an AI-First Product By Dinesh · July 5, 2023 · 11 min read Last updated: July 2023 High user satisfaction in AI products does not come from better models. It comes
AI Engineering
The Expert Matching Problem: When AI Confidence Meets Human Expertise | AI PM Portfolio The Expert Matching Problem: When AI Confidence Meets Human Expertise May 18, 2023 · 15 min read · System Design Case Study At a YC-backed tax-tech startup, we needed to match 16,000 users with the right tax expert
LLMs
LLMs Changed Everything We Thought We Knew About Document Processing | AI PM Portfolio LLMs Changed Everything We Thought We Knew About Document Processing April 10, 2023 · 15 min read · Technical Deep Dive For years, document processing followed a rigid pipeline: OCR to extract text, regex to find patterns, rules to