Hot AI startups are getting by on 'vibes' — but need data as a reality check, says former Facebook VP

The rapid expansion of AI startups has sparked significant discussion about their reliance on intuition rather than solid data foundations. Julie Zhuo, a former vice president of design at Facebook and co-founder of the AI analytics firm Sundial, highlighted this issue, indicating that many of these emerging companies have grown so quickly that they have neglected to establish essential data infrastructure. Zhuo pointed out that it is not uncommon to find startups with just a handful of employees generating hundreds of millions in annual recurring revenue (ARR) and attracting vast user bases. However, she noted that these companies often lack the necessary logging systems to conduct meaningful data analysis. "We see companies that are growing insane, and they're still about 10 people or two people," she remarked, emphasizing the unprecedented pace of growth in the AI sector. Historically, companies did not amass 100 million users overnight. Slower growth processes allowed teams ample time to implement logging systems, recruit data teams, and develop the capability to analyze user behavior and revenue effectively. Zhuo expressed concern that this rapid growth is not sustainable. When the growth rate stabilizes, these startups may find themselves unprepared to address critical questions regarding user retention, feature value, and business drivers. "At that point, that's usually when people start investing a ton in data," Zhuo stated, stressing the importance of data in understanding operational realities. She also emphasized the need to rethink success metrics in the AI context, especially as some companies experience swift growth. For products centered around chatbots or conversational interfaces, new analytical strategies are essential. Instead of traditional metrics like clicks or page views, Zhuo suggested using large language models or machine learning techniques to better comprehend user intent. The rapid growth of AI startups is part of a larger trend, with these companies raising unprecedented funding amounts—over $35 billion in 2024 alone. However, some investors are raising alarms about the potential overheating of the AI market, drawing parallels to the dot-com bubble of 2000. Concerns about the efficacy of large language models and the sustainability of tech companies' extensive investments are prevalent, alongside worries that inexperienced investors may be swept up in the excitement. OpenAI CEO Sam Altman recently described the high valuations of some small AI startups as "insane" and "not rational," expressing his belief that the current investor enthusiasm for AI is excessive, yet he also acknowledged the transformative potential of AI as a critical technological advancement.