Evals are becoming the primary bottleneck for AI progress—they define what success looks like for models and provide the framework for improvement. Without clear evals, companies can't effectively apply AI to automate workflows or measure progress.
"If the model is the product, then the eval is the product requirement document."
Mercor's Unprecedented Growth Story
17
Months
From $1M to $500M revenue
0%
Customer Churn
Never lost a customer
1600%
Net Retention
Exceptional growth rate
6
Magnificent 7
Tech companies served
Mercor became the fastest-growing company in history by solving AI's biggest bottleneck: helping AI labs source experts who create evals and training data. They're working with 6 of the "magnificent 7" tech companies.
Why Models Are Only as Good as Their Evals
"Models are only as good as their evals."
This quote from Brendan's customers captures why this market is exploding. Labs need experts to create rubrics and success criteria across every domain they want models to master.
Without these benchmarks, they can't measure progress or effectively train models. The evaluation framework becomes the foundation for all AI advancement.
The Future of Work: Creating AI Environments
01
Current State
Workers fear AI displacement and job loss across industries
02
Transition Phase
Shift from doing work to defining what good work looks like
03
New Reality
Creating reinforcement learning environments becomes the new job category
Rather than being replaced by AI, many workers will shift to defining what good looks like in their domain. This creates an entirely new category of jobs that hasn't been discussed amid fears of displacement.
Choose Jobs with Elastic Demand
Inelastic Demand
When AI makes people 10x more productive, some industries will simply need 10x fewer workers
Elastic Demand
Fields like software development will build 10x more—creating more opportunities, not fewer
The best jobs to get into now are those with "elastic" demand. Focus on industries where increased productivity drives increased demand rather than job reduction.
Product-Market Fit Reveals Itself
The Struggle
Brendan initially tried to "force" product-market fit through aggressive sales tactics and constant pivoting.
The Breakthrough
Real product-market fit came when customers became surprisingly easy to sell to—the market pulled the product forward.
Be stubborn about your thesis of how the world will change, but flexible about exactly what form your solution takes.
Three Core Values That Drove Growth
Can-Do Attitude
Setting "insane" goals that somehow materialized through sheer determination and belief in possibility
High Standards
Hiring former founders and executives from companies like Uber to maintain exceptional talent density
Intensity
Creating an output-oriented culture where people move heaven and earth to succeed
The First 10 Hires Shape Everything
1
Patient Foundation
Brendan was extremely patient with his first 10 hires, bringing on extraordinary talent like Scale's former head of growth
2
Talent Density
This initial talent density shaped the entire organization as it scaled to unprecedented heights
3
Acceleration Phase
Once the market opportunity was validated, hiring accelerated while maintaining quality standards
Be patient with your first 10 hires, then accelerate once you know the market opportunity is real.
AI Won't Replace You, But...
People Good with AI Will
The most successful people won't be those with specific technical skills but, rather, those who can leverage AI to become dramatically more productive in whatever they do.
Don't fight against AI tools—embrace them and see what you can build with them. The future belongs to those who can amplify their capabilities through artificial intelligence.
Brendan doesn't expect AGI or superintelligence anytime soon. Despite some lab executives predicting superintelligence within three years, he believes it's a longer road.
Models are still terrible at basic tasks like scheduling or using tools effectively. The path forward requires evals for everything models can't yet do—which will take years, not months.
The AI revolution is real, but the timeline is longer than the hype suggests. Focus on building sustainable solutions for today's capabilities.