Workshop: Mastering AI Observability w/Granola & Braintrust

Join us for an engaging hands-on workshop where you’ll dive into the intricacies of AI observability! 🤖✨

Your agent is live in production—you'll gain crucial insights into its performance, pinpoint failures, and strategize improvements.

In this interactive session, we’ll utilize Braintrust to transform production traces into insightful evaluations:

• Leverage Topics to uncover patterns in real agent traces 🔍
• Identify key issues for investigation 🕵️‍♂️
• Create custom scorers to measure those issues directly ⚙️

With this active observability approach, you’ll update your agent, rerun evaluations, and compare results seamlessly within Braintrust. You'll develop a repeatable workflow for translating agent behavior into evaluated metrics, continuously measuring improvements while catching regressions before they affect users. 🚀

What You'll Build:

• A quality analysis using Topics 📊
• A custom scorer for addressing real failure patterns 🛠️
• An enhanced agent complete with traces and evaluation results 🏆

Format: Guided hands-on workshop
Level: Intermediate (prior experience with AI evaluations is necessary)
Bring: A laptop 💻

Location: Four One Nine 📍
Get your tickets here!