I’m going to lead off with some AI downsides.
Judges are increasingly identifying fake legal citations, with a notable rise in misuse attributed to lawyers relying on artificial intelligence tools. Since May 1, judges have reported at least 23 instances of AI-generated inaccuracies in court records, highlighting a growing trend where legal professionals are at fault for these errors. Damien Charlotin, a legal data analyst, has documented 120 cases where courts have recognized AI’s hallucinations—instances where the technology produced fabricated legal quotes or cited non-existent authorities. In a significant shift, data indicates that in 2023, three out of ten cases of AI citation errors were attributed to lawyers, contrasting with the majority of mistakes previously made by individuals representing themselves in court. The implications of these findings are significant, as courts worldwide are imposing fines for AI misuse, with penalties exceeding $10,000 in some cases.
Databricks has revealed that 80% of new databases created on their Neon platform last month were generated by artificial intelligence agents rather than human developers. This trend highlights the growing influence of non-developers utilizing “vibe-coding” platforms, which enable users to create applications by simply providing prompts that AI agents then transform into code and project specifications. The article discusses how these AI-driven tools are changing the landscape for development tools, suggesting that companies must adapt to remain competitive. It emphasizes the importance of optimizing products for visibility to large language models and stresses the need for seamless integration, with companies like Neon and Stripe leading the way in providing user-friendly solutions for AI agents.
A recent article in Futurism highlights the alarming trend of artificial intelligence potentially increasing the gender employment gap, particularly affecting women. According to a report from the United Nations’ International Labour Organisation, the risk of women facing “high automation potential” jobs rose to 9.6 percent in high-income countries like the United States, up from 7.8 percent just two years prior. In comparison, the risk for men is significantly lower at 3.5 percent, which has also increased from 2.9 percent in 2023. The report reveals that one in three workers in wealthy nations face some degree of exposure to automation, with jobs traditionally held by women—such as administrative and clerical roles—being especially vulnerable. Sociologists note that while the gender gap in labor hours has narrowed, the gender pay gap continues to persist, meaning women’s work often shifts towards household tasks rather than formal employment. With the rapid adoption of artificial intelligence, experts warn that without substantial changes in the labor environment, women may face increased challenges in securing stable employment.
Why do we care?
AI’s propensity to hallucinate isn’t a quirky bug—it’s a high-stakes liability, especially when professionals blindly trust its output. The legal profession serves as a proxy for broader regulated industries, such as finance and healthcare, where trust, compliance, and auditability are non-negotiable. For IT service providers supporting clients in these sectors, this is a cautionary tale: any AI deployment must come with robust guardrails, paired with accountability.
If providers focus only on surface-level AI enablement (cheaper ops, faster dev, better marketing), they risk walking clients straight into governance disasters, ethical landmines, and workforce disenfranchisement.
Ignoring these undercurrents may preserve short-term margins—but erodes long-term trust. And in the IT services business, trust is the currency of growth.
And note the shadow IT problem. Procurement and software creation are moving from IT teams to business users wielding AI tools.
Finally, MSPs advising clients on automation strategies need to incorporate a human impact lens, particularly as they assist in transforming business operations. Questions of reskilling, upskilling, and equitable impact should be baked into the conversation—not bolted on later as PR clean-up. There’s also reputational risk for clients (and you) if these impacts are ignored.