Oracle Manipulation

Security in Artificial Intelligence: AI Attack Vectors, Frameworks, and Posture Management

Robotic hand reaching into glowing digital network representing security challenges in artificial intelligence systems

Security in artificial intelligence addresses a problem that emerged faster than the security discipline could adapt: organizations deployed AI systems — LLM applications, agentic workflows, model pipelines — at a pace that outran their ability to assess and manage the unique attack surfaces those systems introduce. 77% of companies experienced breaches in their AI systems in 2025 (Lakera), yet only 5% of organizations report high confidence in their AI security preparedness. The attack surface for AI systems differs fundamentally from traditional application security: prompt injection attacks achieve over 90% success rates against unprotected LLM applications; data poisoning of 0.001% of training tokens increased harmful outputs 4.8x in documented research; 97% of organizations that suffered AI model breaches lacked proper AI access controls (IBM 2025). The frameworks and tools for securing AI — NIST AI RMF, MITRE ATLAS, OWASP Top 10 for LLMs, AI-SPM platforms — are maturing quickly, but deployment of those frameworks lags behind AI deployment by a significant margin.

  • 77% of companies experienced breaches in AI systems in 2025 (Lakera); 97% of breached organizations lacked proper AI access controls (IBM 2025).
  • Prompt injection ranked #1 (LLM01:2025) in OWASP’s Top 10 for LLM Applications; real-world success rates exceed 90% against unprotected systems.
  • 45% of AI-related breaches traced to malware in public model repositories (HiddenLayer 2025); 59% of Hugging Face model files use unsafe pickle serialization.
  • Shadow AI: 38% of employees share confidential data with AI platforms without employer approval; shadow AI incidents cost $670,000 more per breach than standard incidents (IBM 2025).
  • MITRE ATLAS (October 2025): 66 AI attack techniques including 14 new agentic AI techniques; NIST AI 100-2 E2025 published March 2025 with formal AI attack taxonomy.

Scrabble tiles spelling DATA BREACH representing the attack vectors and vulnerabilities that compromise artificial intelligence systems

AI Attack Vectors: How Artificial Intelligence Systems Are Compromised

AI systems introduce attack vectors that do not map cleanly to traditional application security frameworks. A SQL injection attack follows a predictable grammar; a prompt injection attack exploits the semantic flexibility that makes LLMs useful. A supply chain attack against traditional software compromises a package; a supply chain attack against an AI system may compromise a model that was downloaded 50,000 times from a public repository before the payload was detected. The security discipline for AI systems — sometimes called MLSecOps or AI security — is fundamentally about understanding which security assumptions from traditional application development do not transfer to AI, and building compensating controls for the gaps.

OWASP LLM Top 10: Prompt Injection, Data Poisoning, and Supply Chain Risks

OWASP’s 2025 Top 10 for LLM Applications (LLM01:2025 through LLM10:2025) is the authoritative attack taxonomy for LLM-based applications. Prompt injection is ranked first — an attack class where adversarial inputs manipulate an LLM’s behavior by overriding system instructions, extracting sensitive information from system prompts, or hijacking the model’s tool-calling capabilities. Real-world red-team exercises and the October 2025 joint research paper involving OpenAI, Anthropic, and Google DeepMind researchers demonstrated that adaptive prompt injection attacks bypass nearly all published defenses with success rates exceeding 90%. The August 2024 Slack AI incident provides a concrete production example: attackers embedded malicious instructions in channel messages, exploiting the RAG pipeline to exfiltrate data from private channels — combining data poisoning and prompt injection in a real enterprise product.

Data poisoning (LLM04:2025 in the OWASP taxonomy) attacks the training or fine-tuning data that shapes model behavior. Research demonstrates that poisoning an extremely small fraction of training data — 0.001% of medical domain tokens — produces a 4.8x increase in harmful outputs, illustrating that poisoning attacks do not require large-scale data access to cause significant damage. Supply chain risk (LLM03:2025) is the most immediately actionable attack vector for most organizations: 97% of organizations use models from public repositories (HiddenLayer), and 45% of AI-related breaches were traced to malware introduced through those repositories. JFrog’s analysis of 4,023 Hugging Face model repositories found that 59% of serialized model files use unsafe pickle serialization — a format that executes arbitrary code on deserialization. In March 2024, over 100 malicious AI/ML models with payloads including credential theft and reverse shells were discovered on Hugging Face; the platform marks such models as unsafe rather than removing them. Model extraction — where an attacker replicates a proprietary model by querying its API — was documented at scale when OpenAI identified DeepSeek’s unauthorized use of GPT-3/4 API outputs for model distillation in December 2024, resulting in API access revocation. Threat intelligence feeds tracking AI/ML repository compromise indicators provide the earliest available warning of poisoned model distribution.

Shadow AI and Data Leakage: The Internal AI Security Problem

Shadow AI — the use of AI tools and models outside organizational approval and oversight — represents the most operationally immediate AI security risk for most organizations in 2026. Reco AI’s 2025 State of Shadow AI Report found that more than 80% of workers use unapproved AI tools, including nearly 90% of security professionals. CybSafe and the National Cybersecurity Alliance surveyed 7,000 employees and found that 38% share confidential data — including source code, HR records, legal data, and R&D — with AI platforms without employer knowledge or permission. The data leaving through shadow AI is frequently proprietary and often regulated: Samsung engineers pasting proprietary code into ChatGPT queries, lawyers uploading client documents to AI legal research tools, HR staff using AI to analyze compensation data in consumer applications not covered by enterprise data agreements.

The financial impact is measurable: IBM’s 2025 Cost of a Data Breach report documents that shadow AI incidents cost an average of $4.63 million per breach versus $3.96 million for standard breaches — a $670,000 premium. Shadow AI incidents account for 20% of all breaches in IBM’s 2025 dataset. Despite this, 63% of breached organizations either lack an AI governance policy or are still developing one; of those with policies, only 34% perform regular audits for unsanctioned AI use. Gartner projects that by 2030, more than 40% of enterprises will experience security or compliance incidents linked to unauthorized shadow AI — making governance and discovery the operational priority rather than technology controls alone. Security analytics platforms with AI asset discovery capabilities are the foundational tool for shadow AI inventory, but policy and training address the behavioral driver that technology controls cannot fully contain.

USB security key beside laptop representing AI security posture management and access controls for artificial intelligence systems

AI Security Frameworks, Posture Management, and Compliance

The AI security framework landscape consolidated significantly between 2023 and 2026, producing three primary reference structures that most organizations now use in combination: NIST AI RMF for organizational risk governance, MITRE ATLAS for adversarial AI attack taxonomy, and OWASP Top 10 for LLM Applications for application-level security. These frameworks address different audiences — governance teams, security researchers, and application developers respectively — and the overlap between them covers the full AI security lifecycle from model selection through deployment and monitoring.

NIST AI RMF, MITRE ATLAS, and OWASP for LLM Applications

The NIST AI Risk Management Framework (AI RMF 1.0), released January 2023, structures AI risk management across four core functions: Map, Measure, Manage, and Govern. It is the dominant US federal reference for pre-deployment AI security assessments in regulated industries and provides the governance structure for AI risk programs that other frameworks plug into. NIST complemented the RMF with AI 100-2 E2025 (published March 2025) — the updated Adversarial Machine Learning Taxonomy, formally classifying AI attacks into six categories: data poisoning, evasion attacks, model extraction, model inversion, membership inference, and abuse/misuse of generative AI.

MITRE ATLAS (Adversarial Threat Landscape for Artificial Intelligence Systems) is the AI equivalent of MITRE ATT&CK — a structured knowledge base of adversarial tactics and techniques observed against AI systems. The October 2025 ATLAS update documented 15 tactics, 66 techniques, 46 sub-techniques, 26 mitigations, and 33 case studies, including 14 new techniques specifically focused on agentic AI risks: prompt injection into agent memory, tool invocation exfiltration, and memory manipulation. For security teams familiar with ATT&CK-based threat hunting, ATLAS provides the analogous framework for hunting adversarial activity against AI assets. The combined use of NIST AI RMF for governance, MITRE ATLAS for adversarial technique mapping, and OWASP LLM Top 10 for application security creates the comprehensive coverage that no single framework provides alone. Threat intelligence programs that incorporate ATLAS technique tracking provide proactive warning of which AI attack techniques are being actively used against peer organizations.

AI-SPM Platforms and the EU AI Act

AI Security Posture Management (AI-SPM) emerged as a recognized product category by 2024, addressing the need to continuously discover and assess an organization’s AI assets — models, pipelines, agents, applications — for security risk. The AI-SPM market was valued at $4.65 billion in 2024, with Forrester projecting growth to $15.8 billion by 2030. Major CSPM vendors Wiz, Orca Security, Palo Alto Networks (Prisma Cloud), and Tenable all shipped AI-SPM capabilities in 2024–2025. Specialized vendors include HiddenLayer (model scanning, integrated into the Azure AI model catalog), Protect AI (huntr bug bounty platform, ModelScan open-source tool, $700M reported Palo Alto acquisition target), and Lakera (Lakera Guard runtime protection, Lakera Red pre-deployment assessment; acquired by Check Point for a reported $300 million). AWS Bedrock Guardrails’ Automated Reasoning capability claims 99% accuracy for preventing hallucinations using formal logic verification — described by AWS as the first AI safeguard to apply formal logic methods to the hallucination problem.

The EU AI Act, which entered into force August 1, 2024, introduces mandatory security and risk management requirements for AI systems deployed in the EU with the most aggressive penalty structure in the AI governance landscape: up to €35 million or 7% of global annual turnover for deploying prohibited AI practices, and up to €15 million or 3% of global turnover for non-compliance with high-risk AI system obligations. The compliance timeline is enforcement-sequenced: prohibited AI practices from February 2, 2025; GPAI model obligations from August 2, 2025; high-risk AI system requirements from August 2, 2026. For organizations deploying AI in EU markets, the Act creates concrete security documentation requirements — conformity assessments, technical documentation, human oversight implementation, and post-market monitoring — that make AI-SPM platforms a compliance infrastructure investment rather than an optional security tool. Enterprise security intelligence platforms that integrate AI asset inventory with compliance reporting address the documentation requirement that regulated organizations cannot meet with manual tracking.

Frequently Asked Questions

What are the main security risks in artificial intelligence systems?

The main security risks in AI systems according to OWASP’s LLM Top 10 (2025): prompt injection (#1), sensitive information disclosure, supply chain compromise, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. At the organizational level, shadow AI (80%+ of workers use unapproved tools) and AI supply chain risk (45% of AI breaches traced to public model repositories) represent the most widespread operational risks. IBM 2025 data shows 77% of companies experienced AI system breaches, with 97% of those lacking proper AI access controls.

What is prompt injection and why is it the top AI security risk?

Prompt injection is an attack where adversarial inputs manipulate an LLM’s behavior by overriding system instructions, extracting sensitive information, or hijacking the model’s tool-calling and agent capabilities. It ranks first (LLM01:2025) in OWASP’s Top 10 for LLM Applications because it is both extremely common and extremely difficult to fully mitigate — the semantic flexibility that makes LLMs useful is precisely what makes them susceptible. Real-world success rates exceed 90% against unprotected systems, as demonstrated in the October 2025 research paper by OpenAI, Anthropic, and Google DeepMind researchers. The August 2024 Slack AI production incident confirmed that prompt injection can be weaponized at enterprise scale to exfiltrate private data through RAG pipelines.

What is shadow AI and why is it a security concern?

Shadow AI refers to the use of AI tools, models, or services outside organizational approval and oversight — employees using consumer AI platforms (ChatGPT, Claude, Gemini) to process work data, or teams deploying AI tools without security review. It is a security concern because: (1) confidential data enters third-party AI systems without data agreements — 38% of employees share confidential data with AI platforms without employer approval; (2) output from those systems may be used to train public models; (3) unsanctioned AI applications expand the attack surface without corresponding security controls. IBM 2025 documents shadow AI incidents at $4.63 million per breach versus $3.96 million for standard breaches — a $670,000 premium — and attributes 20% of all breaches to shadow AI exposure.

What frameworks exist for AI security and risk management?

Three primary frameworks cover AI security: (1) NIST AI Risk Management Framework (AI RMF 1.0, January 2023) — governance structure using Map/Measure/Manage/Govern functions; the US federal standard for AI risk management; (2) MITRE ATLAS — adversarial AI attack taxonomy analogous to MITRE ATT&CK, with 66 techniques including 14 agentic AI-specific techniques added in October 2025; (3) OWASP Top 10 for LLM Applications (2025) — application security checklist covering prompt injection, supply chain, data poisoning, and 7 additional risks. NIST AI 100-2 E2025 (March 2025) provides the companion adversarial ML taxonomy. Organizations also reference ISO/IEC 42001 for AI management systems and the EU AI Act for compliance obligations.

What is AI Security Posture Management (AI-SPM)?

AI Security Posture Management (AI-SPM) is the continuous practice of discovering, inventorying, and assessing an organization’s AI assets — models, training pipelines, inference endpoints, agents, AI-integrated applications — for security risk and compliance violations. It extends the concept of Cloud Security Posture Management (CSPM) into the AI layer, detecting risks like exposed model endpoints, models trained on sensitive data without access controls, unsanctioned AI tool usage, and models with unverified provenance from public repositories. The AI-SPM market reached $4.65 billion in 2024 (projected $15.8 billion by 2030, Forrester). Major vendors include Wiz, Palo Alto Prisma Cloud, HiddenLayer, Protect AI, and Lakera (Check Point acquisition). For EU AI Act compliance, AI-SPM provides the inventory and documentation infrastructure that high-risk AI system obligations require.