Doximity launches Bedside Bench as founding partner in Fireworks AI index
- Doximity named founding partner in Fireworks’ Specialized Intelligence Index
- Releases open-source Bedside Bench to test clinical-grade AI in real-world scenarios
- Doximity Ask ranked first vs frontier models in Fireworks’ independent grading
- Follows July NOHARM study where Doximity Ask outperformed other US-based models

*this image is generated using AI for illustrative purposes only.
Doximity, Inc. (NYSE: DOCS) has been named a founding partner in Fireworks’ newly launched Specialized Intelligence Index, a cross-industry benchmark for leading AI systems. The digital health platform joins Sierra and Harvey as category leaders spanning healthcare, customer support, and law.
As part of this launch, Doximity released Bedside Bench, an open-source benchmark designed to measure clinical-grade AI. Unlike traditional evaluations that rely on multiple-choice questions to test recall, Bedside Bench utilizes complex clinical scenarios to assess real-life performance and safety at the point of care.
Benchmark methodology and focus areas
The new benchmark targets areas identified by physicians and developers as having the highest impact on medical practice. It moves beyond simple fact retrieval to test an AI’s ability to handle nuanced, high-stakes decisions.
Key test areas include:
- Drug-safety questions where minor errors pose major health risks
- Correct application of evidence from drug guidelines and trials
- Open-ended questions with multiple reasonable answers, differing in completeness and safety
- Prompts containing fabricated drugs or nonexistent studies
- Questions designed to uncover health-equity mistakes and bias
- Ability to select appropriate actions from long lists while avoiding inappropriate ones
Performance against frontier models
When independently graded by Fireworks, Doximity Ask ranked first compared to frontier models. This result follows a previous achievement in July, where Doximity Ask outranked all other U.S.-based models in the independent NOHARM study conducted by researchers at Stanford and Harvard. These outcomes suggest that AI systems built specifically for medicine can exceed general-purpose frontier models on clinically relevant tasks.
Louis-Antoine Mullie, Head of Medical AI at Doximity, stated that clinicians need assurance regarding AI trustworthiness on practical decisions. He noted that Bedside Bench was built around physician-validated, real-world clinical tasks, addressing risks such as drug safety and incorrect use of evidence.
What the numbers show
The dual validation from Fireworks’ independent grading and the July NOHARM study highlights a consistent pattern: specialized medical AI outperforms general-purpose frontier models on domain-specific safety metrics. This divergence suggests that general-purpose scale does not automatically translate to clinical reliability, reinforcing the value of domain-specific benchmarks like Bedside Bench.
How might the adoption of Bedside Bench influence regulatory standards for AI approval in healthcare?
Will Doximity's superior performance on clinical benchmarks translate into measurable revenue growth or increased enterprise contracts?
How are general-purpose AI providers responding to the challenge of specialized medical models outperforming them on safety metrics?
































