Data Engineering in 2026 w/ Zach Wilson

• • Are your Snowflake costs going crazy? Most data teams spend months analyzing Snowflake costs. You can reclaim up to 50% of yours in just hours. Talk to a Revefi expert (https://www.revefi.com/demo)today Thanks to Revefi (https://www.revefi.com/) for partnering on this newsletter. Data Engineering in 2026 w/ Zach WilsonI was on a rooftop in Stockholm last night with Zach Wilson recording an impromptu Freestyle Friday podcast, and somewhere between the sunset and the third spicy take, we hit on a few things worth pulling out of the transcript. This post will be a little different, as I’m stepping out of the limelight and instead sharing some of Zach’s thoughts. Surprise (/s) - AI is Happening Outside of SFZach toured the Lovable office and walked away thinking the Bay should be nervous. The first cafe he stepped into in Stockholm sounded like Hayes Valley - laptops everywhere, the word “agent” tossed around five times in his first hour. Lovable has the revenue numbers to back the energy. The vibe is real. San Francisco has been running the same line for a decade: we’re the only city doing AI right. That story is breaking (I’m also seeing this in various parts of Europe and Asia). Stockholm has the talent, the energy, and (Zach’s words) the mental health that comes from “get rich and maybe relax in the social safety net” rather than “get rich or die trying.” Where to Focus in 2026If you’re a founder or an engineer trying to figure out where to be, the map has more dots on it than the narrative suggests. But the part I want to dwell on is data engineering, since that’s what Zach and I are best known for. Here are some of Zach’s takeaways. First, dashboards are cooked. Those jobs aren’t disappearing tomorrow. But they’re going to pay less, and they’re not where you want to be. Building a Tableau viz so an exec can stare at it for ten seconds is not a future-proof career. The work has moved on. The Three Vs are your moat. Here’s Zach’s framing, and it’s the cleanest I’ve heard. Volume, velocity, variety. The more of these you work with, the safer your role.

• Volume. Big data doesn’t fit in AI’s context windows. If you’re moving petabytes, the model can’t just eat your job for breakfast.

• Velocity. Real-time has too many gotchas. Kafka, ClickHouse, streaming systems, and AI isn’t there yet. Long runway.

• Variety. This is the big one. Rows and columns are the past. PDFs, images, audio, transcripts (like the one this post came from) - that’s the work that matters now. The more your pipelines handle messy, multimodal stuff, the more leverage you have.

If you’re a data engineer working only with low-volume, batch, structured data, you are exposed. Pick a V and go deep. Preferably, stack two of them. For People Just Starting Out - Build Things!I get this question constantly. So does Zach. We landed in the same place. Reading is great, but building is better. Even though I wrote the book, I’ll tell you straight up that books are not enough. Ask Claude to help you build an orchestration tonight. Move some data. Break it. Fix it. See what happens. Fundamentals matter, but they only stick when your hands are on the thing, building it. Thankfully, AI makes it easier than ever to build, iterate, break things, and learn. Why AI is harder to teach than data engineeringThis is something I’ve been chewing on for months, and Zach said it cleanly. AI is hard to teach because the best practices haven’t been established. In data engineering, there are decades of patterns. Clearer right and wrong. In AI, the gap between what actually works and what vendors are selling you is wide, and vendors are very happy to live in it. Be skeptical of anyone who tells you they’ve nailed the right way to prompt, the right way to build agents, the right way to evaluate. We’re all figuring it out in public. That said, according to Zach, two things hold up:

• Specify your prompts. Most people type “fix this” and wonder why the model is bad. Give it 10% more context, and you get 80% better output. This is not subtle. Under-specified prompts are the single biggest skill gap he sees.

• Stop defaulting to Opus or other high-powered models. Using the most expensive model for every one-plus-one task is asking Einstein to do basic arithmetic. The AI will get it right. It will also cost a lot. Match the model to the job. If everyone did this, Zach jokes, Anthropic would lose a billion a year. (Sorry, Anthropic.)

Anyway, this is a very short post because it’s my last day in Stockholm, and I want to go enjoy the beautiful weather in this amazing city. Have a great weekend, Joe Here’s this week’s Freestyle Friday podcast. Available on Spotify, Apple, and wherever else you get your podcasts. Please support the show with a review. It means a lot. Awesome Upcoming EventsHere are a couple of things I’m up to. Much more to come, so stay tuned. • • AI Council The technical conference for humans who ship

For 10+ years, AI Council has been gathering the world’s top AI infrastructure minds to share what’s working (and where we’re headed next). You’re invited for 3 days of high-quality discourse with 1,200+ technical experts, including office hours, small groups, and zero marketing keynotes. Speakers include: The co-inventor of ChatGPT, Creator of DuckDB and Codex, Engineers behind ClickHouse, Databricks, Datadog, and LangChain. Join us May 12-14 in San Francisco. → See the full speaker lineup (http://www.aicouncil.com/) I’ll be there too, but not speaking. If you want to grab a beer with me and see amazing talks from amazing people, register now.

Disclosure: They’re giving me a ticket to attend. AI Council is one of the last indie data/AI events around, and they do great work. I support non-vendor events as much as I can, and so should you. London! I’ll be at Confluent Current London. See you there! When: May 19 to 20 Where: Excel London Register here (https://current.confluent.io/london). Cool Videos and ReadsJosh Wills has spent 25 years writing data pipelines, with a career spanning Cloudera, as Director of Data Engineering at Slack, on the dbt DuckDB adapter, and now training foundation models at Datology AI. He uses coding agents every day. And he keeps running into the same wall: the agents jump to conclusions, fix the wrong thing, and ship pipelines no one understands.

In this conversation, we unpack why AI agents struggle with the messiest, highest-stakes parts of data work, and what it means for the engineers managing them. Here are some things I read this week that you might enjoy.The AI Great Leap Forward (https://leehanchung.github.io/blogs/2026/04/05/the-ai-great-leap-forward/) What the 1920s Can Teach Us About Surviving the AI Revolution (https://www.wsj.com/tech/ai/tech-innovation-changes-1920s-ai-today-be8d3207?st=xk8HF6&reflink=desktopwebshare_permalink) Crashing Waves vs. Rising Tides: Preliminary Findings on AI Automation from Thousands of Worker Evaluations of Labor Market Tasks (https://arxiv.org/pdf/2604.01363) AI Is Forcing CEOs to Make a Stark Choice: Lay Off Workers or Make Them Do More (https://www.wsj.com/tech/ai/ai-is-forcing-ceos-to-make-a-stark-choice-lay-off-workers-or-make-them-do-more-6b1ed771?st=aW7qje&reflink=desktopwebshare_permalink) Mira Murati’s deposition pulled back the curtain on Sam Altman’s ouster | The Verge (https://www.theverge.com/ai-artificial-intelligence/926383/mira-murati-sam-altman-musk-trial-ouster) Opinion | Speak, Yuppie - The New York Times (https://www.nytimes.com/2026/05/04/opinion/yuppies-merit-society-politics-cities.html?unlocked_article_code=1.gFA.kBGI.7oGPZL8x6x0x&smid=url-share) How Everest Has Changed Since Into Thin Air (https://www.theatlantic.com/books/2026/05/whats-changed-since-jon-krakauer-climbed-everest/687019/?gift=ZMZ_s9UT0umHXSy8voU0rXnAwy9AJKJp5F7UNGVKtZ8&utm_source=copy-link&utm_medium=social&utm_campaign=share) Find My Other Content Here📺 YouTube (https://www.youtube.com/@joereisdata) - Interviews, tutorials, product reviews, rants, and more. 🎙️ Podcasts (https://open.spotify.com/show/3mcKitYGS4VMG2eHd2PfDN?si=7bd8ffc4d5b840ce) - Listen on Spotify or wherever you get your podcasts 📝 Practical Data Modeling (https://practicaldatamodeling.substack.com/) - This is where I’m writing my upcoming book, Mixed Model Arts, mostly in public. Free and paid content. If you’re interested in sponsoring my newsletter and podcast, H2 2026 is opening up. Please message me for details. The Practical Data CommunityThe Practical Data Community is a place for candid, vendor-free conversations about all things tech, data, and AI. We host regular events such as book clubs, lunch-and-learns, Data Therapy, and more. 🤖 Join on Discord (https://discord.gg/gNfw5AKWSK)

Thanks for reading! Subscribe for free to receive new posts and support my work.