Microsoft CTO to AI startups: Stop waiting for better models and ‘do the damned experiments’

Microsoft CTO Kevin Scott has some advice for AI startups waiting for the next breakthrough model: the technology can already do far more than most people are getting out of it, so stop waiting and start building. Also: real customer traction still matters more than online buzz. Speaking at a recent South Park Commons event with the organization’s general partner, former Dropbox CTO and Facebook engineer Aditya Agarwal, Scott said founders are sitting on a “gigantic capability overhang” — meaning that current AI systems can do far more than most apps built on top of them. He cited ChatGPT itself as a past example: the underlying model was “pretty old” when it launched, as he put it, and nobody (including Scott and his peers) predicted at the time it would become a potential trillion-dollar product. “The cost of doing the experiments has never been cheaper,” Scott said. “So do the damned experiments. Try things.” The barrier isn’t model capability, he said, but the unglamorous integration work needed to put it to practical use. “Some of the things that you need to do to squeeze the capability out of these systems is just ugly-looking plumbing stuff, or grungy product building,” he said. “But you’re in a startup, that’s kind of your life. It’s more about the grind.” Scott also cautioned founders against mistaking online attention for real traction. The current environment, he said, is flooded with “false signal” — from media coverage to investor interest — that doesn’t really correlate with whether you’ve built something useful. “You’ve got a bunch of people whose business model is getting clicks on articles online or getting people to subscribe to their Substack,” he said. “If you believe the things that particular part of the ecosystem is sending to you in terms of feedback, it could be that you’re steering yourself in exactly the wrong direction.” The real signal, he said, comes from building something customers actually love. Other topics included: Open-source vs. closed-source models (he effectively framed this as a toolbox, not a battle, and said Microsoft uses both). The importance of expert feedback in AI training, which he views as a potential startup advantage. The infrastructure challenge of building memory systems for AI agents, a problem he said won’t be solved by simply training bigger models. See the full talk above or on the South Park Commons Minus One Podcast.