How to Know When Open-Weight AI Models Actually Save You Money

AT&T runs 45 billion AI tokens a day, and open-weight models already handle a quarter of that load. If your business is burning through a frontier-model bill without knowing which tasks actually need frontier intelligence, you're leaving cash on the table every single day. Ross Katz, Principal and Data Science Lead at CorrDyn, works with data teams on exactly this problem: deciding what to build, what to buy, and what to route. He breaks down a Wall Street Journal piece on AT&T's shift to open-weight models and open-weight AI, and tells you where that logic does and doesn't apply to a normal-sized company. Listen and you'll walk away with a practical way to decide when open-weight models are worth the engineering cost, when routing beats self-hosting, and when Chinese-origin models are actually a risk versus just a headline. Ross puts a number on it: if the use case isn't worth more than a million dollars a year, don't bother self-hosting. Ross and Jason get into the real difference between open-source and open-weight models, when a routing layer beats hosting your own infrastructure, and why data exfiltration (not hidden backdoors) is the actual risk with Chinese models. They also cover build-cost estimates for self-hosting, and close with a candid take on the AI bubble debate. This one's for data leaders weighing their AI spend, not for anyone just using ChatGPT subscriptions day to day. Key Takeaways AT&T's chief data and AI officer expects open models to handle up to 80% of their AI workload within a few years, but Ross explains why that math doesn't automatically transfer to a smaller company. Self-hosting an open-weight model isn't just a GPU cost. Ross puts a real number on the engineering talent required, and where the break-even point actually sits. The Chinese-model security story isn't what most headlines suggest. Ross separates the genuine risk (data exfiltration via APIs) from the one that hasn't been proven (hidden backdoors in the weights). There's a specific volume threshold, tied to a proprietary dataset, where a small fine-tuned open-weight model can beat a frontier model outright. Ross names the number of examples it took. Chapter Markers 00:00 AT&T's token numbers and the episode setup 01:05 Are open models ready for large companies 02:38 Why routing captures value other companies leave behind 07:00 Open-source versus open-weight, explained 09:54 How AT&T's smart router actually works 12:11 Shared inference versus dedicated deployment 14:43 The real cost of self-hosting a model 17:19 Where open weights clearly win today 21:35 AT&T's use of Chinese models 22:33 Is the security risk real or overblown 27:32 What smaller businesses should actually worry about 30:57 Questions every data leader should be asking now 41:51 What We're Watching: is AI a bubble Useful Links & Resources Wall Street Journal: AT&T's open-model strategy (the article that prompted this episode) CorrDyn: corrdyn.com Connect With the Show Ross Katz on LinkedIn:   / b-ross-katz   Ross Katz on X: https://x.com/brosskatz CorrDyn on LinkedIn:   / corrdyn   Are you already routing workloads between open and closed models, or is your AI stack still one provider, one bill? Tell us how you're thinking about it. If you want to talk about your data challenges, or you think we got something wrong, find us at corrdyn.com. #EventualConsistency #DataIndustry #AIInfrastructure #OpenWeightAI #BuildVsBuy