A recent article from The Register highlights the growing risks to the software supply chain posed by AI-powered code generation tools. Researchers have discovered that approximately 5.2 percent of package suggestions from commercial models and 21.7 percent from open-source models are non-existent, a phenomenon known as “hallucination.” This situation creates opportunities for malicious actors to exploit these hallucinated package names by uploading harmful software under such fictitious names to package registries, called “slopsquatting”. While commercial tools like ChatGPT-4 exhibited a hallucination rate of around five percent, open-source models had even higher rates. Experts from Socket, an open-source cybersecurity company, warn that this repeatability creates a predictable attack surface, making it crucial for developers to manually verify package names and utilize dependency scanners to enhance security. The only effective way to mitigate this risk is through meticulous verification and testing of AI-generated code in isolated environments before deployment.
A recent article from MIT Technology Review warns that cyberattacks orchestrated by artificial intelligence agents could soon become a reality. Experts like Mark Stockley from the cybersecurity firm Malwarebytes predict that most cyberattacks could be executed by these agents, potentially making them a significant threat this year. While current cybercriminals have not yet deployed AI agents on a large scale, researchers have shown that these agents can carry out complex attacks. The LLM Agent Honeypot project has logged over 11 million attempts to access vulnerable servers, identifying eight potential AI agents from Hong Kong and Singapore. Furthermore, research indicates that agents could exploit vulnerabilities in systems up to twenty-five percent of the time when given a brief description of the weaknesses. As AI continues to develop, experts emphasize the need for proactive measures in cybersecurity to prevent what could be a substantial increase in agent-driven cyber threats.
Why do we care?
AI-generated code is not merely a convenience; it represents a new attack surface. The hallucination problem in code suggestions—especially the inclusion of non-existent packages—is not just a bug; it is an exploit vector.
The commercial model hallucination rate of 5.2% might seem small—until you realize that a developer copying code without vetting it has a 1-in-20 chance of introducing a fictitious dependency into production.
Now, Let’s not overhype the risk. The slopsquatting exploit requires developers to act recklessly—copy-pasting code and installing dependencies without verification. That’s not good practice, and it’s manageable through policy, tooling, and awareness.
However, with “vibecoding” becoming a trend, the ability of untrained developers to deploy code poses a real risk. This isn’t just a technical problem; it’s a trust issue. IT services firms that can provide assurance—verifiable, documented assurance—that their AI-enhanced tools do not introduce new risks will be the ones who thrive. The others will be left cleaning up avoidable messes.