High model performance, on average, can hide that models may systematically underperform on subgroups of the data. To tackle this, we propose Data-IQ, a framework to systematically stratify examples into subgroups with respect to their outcomes — allowing users to audit their tabular, image or text data with just two lines of extra code! We do this by analyzing the …

Instructional videos depict humans demonstrating how to perform multi-step tasks such as cooking, repairing, etc. Building good video representations from instructional videos is challenging due to the small amount of video annotations available. This makes extracting the procedural knowledge such as the identity of the task (e.g., ‘make latte’), its steps (e.g., ‘pour milk’) challenging. Our insight is that instructions …

Large multimodal datasets have been instrumental in recent breakthroughs such as CLIP, Stable Diffusion, and GPT-4. At the same time, datasets rarely receive the same attention as model architectures or training algorithms. To address this shortcoming in the ML ecosystem, we introduce DataComp, a benchmark where the training code is fixed and researchers innovate by proposing new training sets. We …

Large language models are fluent text generators, but they struggle at generating factual, correct content. How can we convert these capabilities into reliable, production-grade applications? In this talk, I’ll cover several techniques to do this based on my work and experience at Stanford and Databricks. On the research side, we’ve been developing programming frameworks such as Demonstrate-Search-Predict (DSP) that reliably …

Join us for a fireside chat with Gideon Mann, Head of Machine Learning Product and Research, CTO Office at Bloomberg about how they built a domain-specific LLM, BloombergGPT.

Discover the transformative power of data in developing Stable Diffusion and Generative AI as Emad Mostaque shares insights into the pivotal role data plays in creating these groundbreaking technologies. Explore the journey of leveraging data-driven approaches to drive innovation, unlock new possibilities, and shape the future of AI.

In this illuminating fireside chat, we dive into the heart of modern enterprise AI, exploring the dynamic intersection of data, models, and MLops platforms that define the new ML stack. We’ll investigate how factors such as model form factors, data types, use case variety, enterprise constraints, and the use of private data in AI applications shape this landscape, all while casting …

Join us for an engaging fireside chat as we delve into data science’s history, impact, and challenges in the United States government. Our esteemed guest, the former U.S. Chief Data Scientist, will share insights into the origins of this vital role and their experiences in managing critical initiatives. Discover the strategies to drive data-driven decisions within the complex governmental landscape …

In this session, we’ll dive into the intricacies of Large Language Models (LLMs) within regulated industries. Our expert panel will discuss strategies for tuning LLMs to reduce misinterpretations and errors in conversational AI applications, emphasizing the necessity of precision in such sectors. They’ll explore the challenges and potential solutions organizations might encounter when transitioning from rules-based approaches to LLMs. Further, …