You Kaichao: vLLM, Open-Source Infra, Model Co-Design & Journey from Community to Startup

Today, our guest is You Kaichao, co-founder and Chief Scientist of Inferact — a remarkable startup that traces its roots to vLLM, a campus open-source project born at UC Berkeley. Over nearly three years, the project's community maintainers transformed it from an academic algorithm paper into a thriving open-source community, and eventually into a sustainably operating company. These maintainers resisted immense personal temptations, steadfastly upholding the open-source spirit they believed in. Earlier this year, Inferact closed a $150 million seed round. We discussed the dilemmas, transformations, and reflections that emerge when an open-source project, imbued with deep community passion, evolves into a commercially operated organization. At the same time, we wanted to bring fresh perspectives from the cutting edge of AI infrastructure. So I also spoke with You Kaichao about how multi-party co-design should be approached in an era where AI infrastructure, model architectures, and even hardware engineering must be designed in concert. Here is my interview with You Kaichao. --- 对游凯超3小时访谈：开源Infra、模型Co-design 、“如果vLLM失败，我们会后悔一辈子” 今天我们的嘉宾是游凯超，他是创业公司Inferact联合创始人兼首席科学家——这个公司很特殊，它是从伯克利一个校园开源项目vLLM演化而来。这个社区的维护者用了将近3年时间，把它从一篇算法论文变成了一个开源社区，又变成了一个公司的正常运营。 这些维护者们放弃了大量的个人诱惑，坚持着他们所相信的开源精神。 今年初，Inferact获得了1.5亿美元的种子轮融资。我们讨论了当一个寄托着社区情怀的开源项目，变成一个商业化的组织运营之后，它所面临的抉择、转变与思考。 另外一方面，我们也非常希望在AI Infra的技术前沿上，给大家带来一些新的思路。所以我也和游凯超聊了聊，在AI Infra、模型结构，甚至Harness Engineering都要联合设计的时代背景下，多方的联合设计应该怎么做？ 接下来，就是我对游凯超的访谈。 --- OUTLINE: 00:02:18 From Algorithms to Machine Learning Systems——从算法到机器学习系统 00:37:56 The Birth of the Open-Source Project vLLM——开源项目vLLM的诞生 01:07:25 "If vLLM Failed, We Would Regret It for the Rest of Our Lives”——“如果vLLM失败了，我们会后悔一辈子” 01:20:11 "Benevolent Dictator”——“仁慈的独裁者” 01:37:16 From Community to Startup——从社区到创业 01:52:56 Co-Design of Models and Infrastructure——模型与Infra的Co-design 02:15:10 Tokens vs. Electricity——Token VS 电力 02:35:41 Technology Predictions——技术预测 --- DISCLAIMER: This content does not constitute investment advice. CONTACT: xiaojunzhang@lisw.ai Jump into the new world — and explore with us! 😉