Help define how
we evaluate AI.
Benchmarking and Evaluating AI Systems in High-Stakes Domains
Bring your research to a focused conversation on AI safety, reliability, and real-world impact. Students and early-career researchers are invited to submit.
250 words
Due September 25, 2026
Saturday, November 7, 2026Day 3, 1:30-3:30 PM PT
AIAS+ conferenceNovember 5-7, 2026
San FranciscoThe Ritz-Carlton
Present your work
Accepted abstracts become posters, with selected submissions invited for spotlight talks.
Develop your ideas
Get feedback and exchange approaches with researchers working across consequential uses of AI.
Bring work in progress
Preliminary ideas and ongoing studies are welcome alongside recently completed research.
A global supporter of brain science and AI research, whose founders established the Chen Institute for Neuroscience at Caltech with a $115 million gift. This workshop brings that scientific ambition to the evaluation of AI.
About the instituteWhat are you
working on?
If your work helps us understand how AI performs when the consequences matter, we want to hear about it.
Benchmark design
New datasets and evaluation protocols for healthcare and other high-stakes domains; benchmarks for generative AI and LLMs; standardization and reproducibility.
Evaluation beyond accuracy
Reliability, robustness, calibration, and uncertainty; distribution shift and generalization; longitudinal and real-world validation; human-in-the-loop and expert evaluation.
Safety, interpretability, and fairness
Safety and failure analyses, stress testing and adversarial evaluation, interpretability and explainability, bias and subgroup performance.
Evaluation infrastructure and security
Evaluation pipelines, privacy and access control, anti-gaming approaches, open versus closed benchmarks, hidden test sets, and API-based evaluation.
Methods, applied validation studies, and methodological reflections are all welcome.
Start with 250 words.
Tell us what you are investigating, your approach, and why it matters for evaluating AI.
- Who
- Students and early-career researchers
- Include
- Title, primary author name and email, and a 250-word abstract
- Deadline
- September 25, 2026Reviewed on a rolling basis
How abstracts are selected
We consider clarity and relevance, methodological contribution, practical importance, and diversity of domains, perspectives, and evaluation challenges.
Student Sponsorship / Financial Support for Registration Limited support from Kivira Health is available for accepted student presenters, subject to approval and availability. See terms.
Poster submission is separate from event registration. Accepted presenters must also register through AIAS+.
Two hours.
Ideas worth discussing.
Saturday, November 7, 2026 (Day 3), 1:30 PM-3:30 PM
Part of the broader November 5-7 AIAS+ symposium
Dong-han Yao, MD
Applied generative AI, healthcare evaluation, and responsible deployment.
Research, in focus
Up to five selected submissions, each with a seven-minute talk and two minutes of questions.
Conversation around the work
Approximately ten posters in a discussion forum for feedback, collaboration, and exchange.
Coming to listen, learn, and connect?
Register to attend via AIAS+