Agenda

Large language models are fluent text generators, but they struggle at generating factual, correct content. How can we convert these capabilities into reliable, production-grade applications? In this talk, I'll cover several techniques to do this based on my work and experience at Stanford and Databricks. On the research side, we've been developing programming frameworks such as Demonstrate-Search-Predict (DSP) that reliably connect an LLM to factual information and automatically improve the app's performance over time. On the industry side, Databricks has been building a stack of simple yet powerful tools for "LLMOps" into the MLflow open source framework.