AI is creating a security problem most companies aren't staffed to handle, says an AI researcher

As artificial intelligence continues to evolve, a growing number of startups are emerging to address the unique security challenges it presents. However, many organizations are still ill-equipped to manage the complexities of AI system failures, according to Sander Schulhoff, a prominent AI security researcher. During a recent episode of "Lenny's Podcast," Schulhoff emphasized that while companies may have established cybersecurity teams, they often lack the necessary expertise to identify and mitigate risks associated with AI technologies. Traditional cybersecurity professionals are trained to fix known vulnerabilities, but AI systems operate differently. "You can patch a bug, but you can't patch a brain," Schulhoff remarked, highlighting the fundamental disconnect between traditional security approaches and the behavior of large language models. This gap becomes evident in practical applications. For instance, cybersecurity experts might evaluate an AI system for technical flaws without considering the potential for manipulation through deceptive language or indirect commands. Schulhoff, who leads a prompt engineering platform and organizes AI red-teaming hackathons, pointed out that those knowledgeable in both AI security and traditional cybersecurity are better positioned to respond to situations where an AI model generates harmful content. They are likely to isolate the generated code in a secure environment, ensuring that it does not compromise the overall system. Schulhoff also critiqued many AI security startups, stating that some are promoting protective measures that do not provide genuine safety. Given the numerous ways AI systems can be exploited, claims that certain tools can "catch everything" are misleading. He warned of an impending market correction where the profitability of these automated security solutions could diminish significantly. The surge of interest in AI security startups is evident, with major tech companies and venture capitalists investing heavily. For example, Google recently acquired cybersecurity startup Wiz for $32 billion to enhance its cloud security capabilities. CEO Sundar Pichai acknowledged that AI is introducing "new risks" in a landscape increasingly characterized by multi-cloud and hybrid environments. As organizations seek effective cybersecurity solutions, the demand for tools that can monitor and secure AI systems is likely to continue growing.