From React to Dagster: Pete Hunt on Data, Infra, and AI-Ready Platforms
Is Postgres actually a better message queue than Kafka? This provocative question is just one of many insights Pete Hunt shares in this conversation about data orchestration, platform engineering, and the evolution of infrastructure. Pete Hunt, CEO of Dagster Labs and former React co-founder at Facebook, brings his unique perspective from working at tech giants like Instagram and Twitter to discuss how different platform team approaches impact product development. Having witnessed both Facebook's clear delineation between product and infrastructure teams and Twitter's DevOps-style ownership model, Pete offers valuable comparisons of these contrasting philosophies. The conversation explores: How Dagster provides a higher-level abstraction for data teams, making it easier to track and debug data assets rather than just managing workflows The challenges of modern data platforms and why many organizations struggle with complex, distributed systems that could be simplified A practical approach to migrating from Airflow to Dagster with their "Airlift" toolkit that allows for incremental, low-risk transitions How AI development is fueling demand for better data orchestration as companies…
On this episode
Nobody's credited on this episode yet.