Introduction: We Were Never Good At This
The technology industry loves to tell itself a story.
It is a story about innovation, intelligence, and progress. A story where each generation of technology solves the problems of the last. A story where better tools inevitably create better outcomes.
Security history tells a different story.
For decades, the greatest threat to computer systems has not been elite hackers, sophisticated malware, or nation-state actors.
It has been people.
Forgotten patches.
Misconfigured servers.
Hardcoded credentials.
Exposed databases.
Overly permissive access controls.
Ignored warnings.
The weakest point in every system has always been the human being responsible for operating it.
This reality has been demonstrated repeatedly throughout the history of computing. From small businesses to multinational corporations, the root causes of catastrophic security failures are often surprisingly mundane. They are not failures of technology. They are failures of understanding.
Now the industry has embraced a new paradigm.
Generate first. Understand later.
Or perhaps more accurately, generate first and hope understanding is unnecessary.
Generative AI promises to make software development faster, cheaper, and more accessible. What it does not promise—and what it cannot provide—is expertise. Yet expertise is precisely what security depends upon.
The result is an industry that is rapidly increasing its ability to produce software while simultaneously decreasing the incentives required to understand it.
That should concern everyone.
Because security has never depended on how much code exists.
It depends on how many people understand it.
The Want of Understanding
Security has never been primarily a technical problem.
It is an understanding problem.
The most successful attacks in history often exploit failures that were already known, already documented, and in many cases already solved. The issue is not that organizations lack security tools. It is that they lack sufficient understanding of the systems those tools are intended to protect.
This “want of understanding” appears repeatedly throughout the industry.
Consider the common causes behind major breaches:
- Unpatched software despite available fixes.
- Administrative interfaces exposed to the internet.
- Excessive user permissions granted for convenience.
- Security controls disabled to avoid operational friction.
- Developers deploying code they do not fully understand.
- Management treating security as a compliance exercise rather than a risk management discipline.
None of these failures require extraordinary attackers.
They require opportunity.
Historically, the complexity of software acted as a natural barrier. Building a large system required years of accumulated knowledge. Developers learned through mistakes, incidents, debugging sessions, and production failures. The process was slow, but it produced understanding.
Generative AI changes the equation.
A developer can now generate authentication systems, deployment pipelines, cloud infrastructure configurations, and entire application frameworks with little understanding of how they function internally. The code compiles. The application runs. The ticket closes.
The knowledge never arrives.
This distinction matters because security does not emerge from working software.
Security emerges from understanding how software fails.
HyperVM: When Small Mistakes Become Catastrophes
The story of HyperVM remains one of the most sobering examples in technology history.
Developed by LxLabs, HyperVM was a popular virtualization management platform used by hosting providers around the world. In 2009, attackers exploited a vulnerability in the platform, compromising thousands of servers and causing widespread disruption across the hosting industry.
The technical details matter.
The human consequences matter more.
The incident triggered enormous operational and reputational damage. Customers lost confidence. Systems failed. Businesses were disrupted. The pressure on those responsible became immense.
Shortly afterward, LxLabs founder and CEO K.T. Ligesh was found dead in what authorities ruled a suicide.
It is impossible to discuss the incident without acknowledging the tragedy involved.
Too often, security discussions reduce failures to statistics. We count compromised records, financial losses, and downtime metrics. We rarely discuss the people left carrying the consequences.
HyperVM serves as a reminder that software failures are not abstract.
They affect careers.
They affect companies.
They affect lives.
The lesson extends beyond a single vulnerability. Complex systems fail when understanding fails. Rarely does a catastrophic breach result from a single isolated mistake. More often it emerges from layers of assumptions, shortcuts, blind spots, and overlooked risks accumulating over time.
This is precisely why the industry’s current obsession with AI-assisted development should concern us.
We are creating more complexity.
We are generating more code.
We are accelerating deployment.
But we are not increasing understanding at the same rate.
In many cases, we are reducing it.
Equifax and the Myth of the Advanced Attack
If HyperVM demonstrated the human cost of security failure, Equifax demonstrated the organizational cost.
The 2017 Equifax breach exposed sensitive information belonging to approximately 147 million individuals. Names, Social Security numbers, birth dates, addresses, and other personal information were compromised.
For many people, the scale of the incident implied sophistication.
Surely an attack of that magnitude must have involved extraordinary technical capabilities.
The reality was considerably less dramatic.
The vulnerability exploited in the attack was known.
A patch already existed.
The fix was available.
The failure occurred anyway.
This is one of the most important lessons in cybersecurity:
Most catastrophic breaches are not caused by advanced attacks.
They are caused by ordinary negligence operating at extraordinary scale.
Organizations often imagine security as a battle against increasingly sophisticated adversaries. In reality, many attackers simply exploit vulnerabilities that defenders already know about but failed to address.
AI threatens to amplify this problem.
A developer who does not understand a generated dependency chain cannot assess its risks. A team deploying AI-generated infrastructure may not recognize insecure configurations. An organization moving faster than it can review its own systems creates additional opportunities for failure.
The attacker’s job becomes easier when defenders no longer understand the systems they are defending.
The Security Debt Factory
The software industry speaks frequently about technical debt.
Security debt receives far less attention.
Every shortcut introduces future risk.
Every unreviewed dependency introduces future risk.
Every copied code snippet introduces future risk.
Every generated function introduces future risk.
AI dramatically increases the rate at which organizations can accumulate this debt.
A junior engineer might once have spent several hours implementing a feature incorrectly. Today, that same engineer can generate dozens of implementations before lunch. The output appears professional. The architecture appears coherent. The code often appears correct.
Appearances are not security.
Verification remains the bottleneck.
Human review remains the bottleneck.
Understanding remains the bottleneck.
Organizations are discovering that code generation scales extremely well. Security validation does not.
The result is an industry capable of producing vulnerabilities faster than it can identify them.
The debt accumulates quietly.
Then one day it arrives all at once.
Security Theater at Machine Speed
One of AI’s most dangerous capabilities is not generating software.
It is generating confidence.
Modern organizations rely heavily on documentation to assess security maturity. Risk assessments, security reviews, architecture diagrams, compliance reports, policy documents, and audit artifacts all serve important functions.
Historically, producing these materials required substantial effort and expertise.
Today they can be generated in minutes.
This creates a dangerous illusion.
Organizations can now generate evidence of security activity faster than they can perform security activity itself.
Consider how easily AI can create:
- Security policies.
- Risk assessments.
- Compliance documentation.
- Threat models.
- Incident response procedures.
- Architecture reviews.
The resulting documents often look professional.
They frequently sound authoritative.
They may even be technically accurate.
What they do not guarantee is that any of the underlying work has actually occurred.
Attackers do not compromise policy documents.
They compromise systems.
A perfectly formatted security report cannot compensate for an exposed database. An AI-generated threat model cannot compensate for poor access controls. A compliance dashboard cannot compensate for engineers who do not understand the systems they operate.
The industry risks creating a future where documentation quality improves while actual security quality declines.
The reports look better.
The audits look better.
The presentations look better.
The systems remain vulnerable.
This is security theater operating at machine speed.
The Feedback Loop of Failure
Perhaps the most concerning long-term security problem introduced by AI is the possibility of recursive failure.
Large language models learn from existing code.
Existing code contains vulnerabilities.
It contains insecure patterns.
It contains architectural mistakes.
It contains decades of accumulated bad habits.
From the perspective of machine learning, insecure code is still code.
A flawed implementation repeated thousands of times becomes statistically significant regardless of whether it is actually correct.
This creates a dangerous cycle.
- Existing insecure code becomes training data.
- AI reproduces similar patterns.
- Developers deploy those patterns.
- The resulting code enters public repositories.
- Future models train on those repositories.
- The cycle repeats.
Historically, bad engineering practices spread slowly through organizations, mentorship, conference talks, and copied examples.
AI changes the scale completely.
A flawed authentication pattern can now be reproduced thousands of times across unrelated projects by developers who may not possess the expertise required to recognize the problem.
The pattern spreads not because it is secure.
It spreads because it is common.
This is one of the least discussed risks associated with generative AI.
Not that it invents entirely new vulnerabilities.
But that it industrializes existing ones.
The Illusion of Security Expertise
The most dangerous vulnerability may not be technical at all.
It may be psychological.
AI is remarkably effective at producing the appearance of expertise.
Generated explanations sound authoritative.
Generated code looks professional.
Generated architecture diagrams appear thoughtful.
Generated security recommendations often resemble those produced by experienced practitioners.
The illusion is compelling.
The understanding behind it frequently does not exist.
A security engineer develops intuition through years of exposure to failure. They learn to recognize dangerous assumptions. They develop an instinct for identifying risks that are not immediately visible.
That intuition cannot be generated.
It must be earned.
Yet AI increasingly allows organizations to create the appearance of expertise without investing in the expertise itself.
This distinction becomes critical during incidents.
When systems fail, organizations do not need generated answers.
They need people who understand why those answers are wrong.
Security Without Understanding
The technology industry often treats security as a tooling problem.
Buy another scanner.
Deploy another platform.
Install another monitoring solution.
Add another AI assistant.
The assumption is that enough automation can compensate for insufficient expertise.
History suggests otherwise.
Every major security incident eventually returns to the same root cause.
Someone failed to understand something important.
No vulnerability scanner can compensate for ignorance.
No AI assistant can compensate for expertise that was never developed.
No automated review system can replace engineers who understand how systems fail.
Security begins and ends with understanding.
Everything else is support infrastructure.
The industry is investing billions of dollars into generating software faster.
Far less attention is being paid to generating understanding.
That imbalance should worry us.
Because software can survive bad code.
It cannot survive the disappearance of the people capable of recognizing it.
The Coming Security Dark Age
The greatest threat facing modern security is not artificial intelligence.
It is the growing belief that understanding is optional.
Organizations increasingly measure velocity rather than comprehension. Features rather than expertise. Output rather than capability.
The consequences may not be immediately visible.
For a time, everything appears to work.
Applications launch.
Infrastructure scales.
Tickets close.
Reports look good.
The vulnerabilities accumulate beneath the surface.
Then the incident arrives.
And nobody knows why.
Because nobody truly understood the system in the first place.
The technology industry may eventually discover that its most valuable security asset was never a scanner, a compliance framework, or an AI model.
It was the engineers capable of understanding what the software was actually doing.
By the time that realization arrives, a generation of expertise may already have been lost.
Conclusion
The dark ages were not dark because people lacked tools.
They were dark because knowledge was lost.
That is the danger confronting modern security.
Not that AI will suddenly make every system vulnerable.
But that it may gradually reduce the amount of expertise available to identify, understand, and prevent vulnerabilities in the first place.
Security has always depended upon understanding.
The tragedy of the AI era may be that understanding is increasingly treated as an unnecessary expense rather than a strategic necessity.
Code can be generated.
Documentation can be generated.
Policies can be generated.
Expertise cannot.
And a world that forgets that distinction may discover, too late, that it has automated away the very thing that kept its systems secure.