The EU AI Act Readiness is an Evidence Problem for AI Security Teams

The EU AI Act is often treated as a legal and compliance project. That makes sense. It is a law, after all.

But for companies developing or using high-risk AI systems, the challenge is not limited to writing policies. They also need to show that those policies are backed by real testing, controls, monitoring, and records.

Can the organization show how the system was tested? Can it explain what happened when the system broke a rule? Can it prove which control was triggered, what action was taken, and whether the issue was fixed?

That is where the tidy policy document meets the much less tidy reality of running AI.

The EU AI Act places requirements on high-risk AI systems across risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy, robustness, cybersecurity, and post-market monitoring. These duties are spread across legal, compliance, product, engineering, risk, security, and business teams.

Still, when someone asks for technical proof, the security team will probably get the call. Lucky them.

Compliance needs more than a policy

A policy may say that an AI system is monitored. That is a reasonable start, but it is not much help when an auditor asks what was monitored, what was found, and what happened next.

A risk manager may want to know whether sensitive data was exposed. Leadership may ask whether the system ignored a policy. A regulator may ask which controls were active at the time.

At that point, “we take AI safety seriously” is not evidence. It is marketing copy.

The need for evidence appears throughout the Act. Under Article 11, providers of high-risk systems must prepare technical documentation before the system is placed on the market or put into service, and keep that documentation current. Annex IV adds detail, including information about capabilities, limits, testing procedures, risk controls, and monitoring arrangements.

Record-keeping gets its own requirement too. Article 12 states that high-risk AI systems must technically allow for the “automatic recording of events (logs)” throughout the system’s lifetime.

In practical terms, organizations need records showing what happened, when it happened, which system was involved, which policy was triggered, and what action followed.

Useful evidence can include the tests run against an AI application or agent, the weaknesses found, the prompt or response involved, the policy decision made, and whether content was blocked, redacted, replaced, or flagged. It should also show what was fixed and whether the system passed later testing.

That last part matters because AI systems do not stay still. Models change. Prompts change. Retrieval sources change. Agents receive new tools. Employees find new ways to use AI, some clever and some rather less so.

A readiness program built around one review at launch can become outdated surprisingly fast.

Risk management cannot stop at launch

The risk management system required under Article 9 is not supposed to be a one-time exercise. The Act describes it as a “continuous iterative process” that should continue across the system’s lifecycle. It must cover known and reasonably foreseeable risks, including risks that may appear during real use.

That makes one-time testing difficult to defend.

A system that passed an assessment six months ago may be very different today. A new model, data source, tool, integration, or workflow can introduce a fresh attack path.

DeepKeep AI Red Teaming helps test AI applications and agents before and after deployment. It can assess risks such as prompt injection, jailbreaks, data leakage, unsafe outputs, manipulation, goal hijacking, and agent or tool abuse.

The result is not just a list of failures. Teams receive evidence showing what was tested, how the system responded, which risks were found, and what should be fixed.

This supports the lifecycle approach required under Article 9, together with the technical documentation and quality processes described in Articles 11 and 17.

For a closer look at the testing process, see DeepKeep’s guide to AI red teaming. Because AI security testing should be an ongoing process, not a launch-day ritual followed by collective amnesia.

Test for data and fundamental rights risks

Data quality and governance form another major part of the Act. Article 10 covers issues such as relevance, representativeness, errors, gaps, and possible bias in the data used to develop high-risk systems.

For certain high-risk deployments, Article 27 also requires a fundamental rights impact assessment before the system is used. This assessment should consider affected groups, possible harms, human oversight, and the measures taken to reduce risk.

DeepKeep AI Red Teaming can test system behavior against an organization’s datasets, policies, and risk taxonomies. This can help surface privacy exposure, biased behavior, manipulation, unsafe responses, and weak oversight.

The AI Firewall can then enforce policies around sensitive data, restricted topics, harmful content, and other risks during live use.

DeepKeep does not perform the legal impact assessment for the organization. It provides technical findings and control evidence that can support it.

Document what the system can and cannot do

Technical documentation is useful only when it reflects how the system actually behaves.

The requirements under Article 11 call for documentation covering the design, purpose, performance, limitations, testing, and risk controls of a high-risk system. Article 13 adds a transparency duty, requiring enough information for deployers to understand the system’s capabilities, limitations, expected performance, and risks.

This creates a practical question: how do you document limitations that have never been properly tested?

DeepKeep AI Red Teaming shows how systems behave under normal, adversarial, and policy-breaking conditions. It can reveal where the system follows instructions, where it fails, and where controls need improvement.

Model Scanning adds evidence about the AI model itself, including security, integrity, licensing, and supply chain findings. The Agent Attack Surface Scanner examines what an agent can access, what it can trigger, and where dangerous paths may exist.

Together, these capabilities produce artifacts such as test results, attack traces, scanning findings, risk reports, and remediation guidance. These can support the technical records expected under Article 11 and Annex IV.

This is especially important for agents. An agent may have access to email, files, databases, APIs, or business systems. Knowing what the agent can do matters just as much as knowing what the model can say.

An agent with broad access and weak controls is not quite the productivity breakthrough the slide deck promised.

Make human oversight possible

Effective human oversight requires more than placing a person somewhere in the process and hoping they notice when things go wrong.

Under Article 14, high-risk AI systems must be designed so that people can understand their behavior, recognize problems, interpret outputs, and intervene when needed.

That is difficult when the organization cannot see what the system is doing.

DeepKeep gives teams visibility into risky behavior, policy violations, attack paths, and repeated problem patterns. Security and governance teams can review findings, tune policies, investigate events, and intervene where needed.

AI Red Teaming helps reveal the conditions under which oversight may fail. The AI Firewall and AI Lens then provide visibility and policy control during actual use.

The goal is not to place a human behind every prompt, staring at a dashboard until morale improves. It is to give the right people enough information and control to act when it matters.

Protect accuracy, robustness, and cybersecurity

The link between the EU AI Act and AI security becomes especially clear in Article 15.

High-risk systems must achieve appropriate levels of accuracy, robustness, and cybersecurity throughout their lifecycle. They must also be resilient against attempts by unauthorized parties to alter their use, outputs, performance, or behavior.

DeepKeep AI Red Teaming tests whether applications and agents can be manipulated through prompt injection, jailbreaks, indirect attacks, unsafe tool use, and other adversarial techniques.

Model Scanning helps identify risks before models are trusted. The Agent Attack Surface Scanner helps uncover exposed tools, permissions, and workflows.

During production use, the AI Firewall inspects prompts, responses, retrieved context, multimodal inputs, applications, agents, and tool calls. Depending on the risk, it can block the interaction, redact sensitive data, replace unsafe content, refine the input, or alert the security team.

This allows organizations to move from saying the system is protected to showing which protection acted, why it acted, and what happened next.

Keep logs that are actually useful

High-risk systems must support automatic logging under Article 12, while Article 19 requires providers to retain automatically generated logs under their control for an appropriate period.

Not all logs are equally useful, of course.

A timestamp saying that “something happened” is technically a record. It is also how incident responders lose the will to live.

DeepKeep records security-relevant information such as AI interactions, policy decisions, violations, guardrail actions, attack attempts, testing outcomes, and scanning findings.

This helps teams reconstruct what happened and answer practical questions. Which system was involved? Which policy was triggered? Was data exposed? Was the interaction blocked? Did the same issue happen before? Was the system tested again after remediation?

These records can support technical documentation, traceability, oversight, investigation, and reporting.

Monitor AI in real use

Providers are not the only ones with obligations. Deployers of high-risk AI systems also have responsibilities.

Article 26 requires deployers to follow the provider’s instructions, assign human oversight, monitor the system’s operation, and retain logs under their control. A useful overview of these and other high-risk requirements is available in the EU AI Act Explorer.

This is where AI Lens and the AI Firewall become especially relevant.

AI Lens helps security teams understand how employees and developers use AI across the organization. It supports usage visibility, monitoring, investigation, and policy enforcement. The AI Firewall applies controls during live interactions and records the decisions made.

Together, they help deployers see whether AI is being used as expected, detect misuse, apply policies, and preserve evidence.

The aim is not to block every AI tool and send everyone back to manual work. Nobody wants that, including the people writing the policies.

The aim is to allow useful AI adoption without losing control of data, risk, and accountability.

Monitor after deployment and prepare for incidents

The EU AI Act does not treat deployment as the finish line.

Post-market monitoring under Article 72 requires providers to actively collect and analyze relevant data about the performance of high-risk systems throughout their lifetime. Serious incidents must also be reported under Article 73 within the required timeframes.

Proper reporting depends on being able to detect, investigate, and document what happened.

Organizations should therefore be ready to answer some uncomfortable but reasonable questions. Did the system remain within its expected limits? Did new risks appear? Were controls enforced? Were incidents detected? Were logs preserved? Were findings reviewed and fixed?

DeepKeep supports this operating model through runtime monitoring, AI usage visibility, repeated red teaming, model rescanning, policy enforcement, and security logging.

These controls do not make compliance automatic. Nothing does, despite what some dashboards may strongly imply.

They make the technical facts easier to produce, review, and defend.

From compliance claims to defensible evidence

The practical EU AI Act question is not simply whether the organization has an AI policy. It is whether the organization can answer technical questions with proof.

When legal, audit, risk, or leadership asks how an AI system behaved, the security team should be able to show what was tested, what was found, what was monitored, what was controlled, what was logged, and what changed.

DeepKeep helps close the gap between written AI governance and real AI operations. It supports risk management under Article 9, technical documentation under Article 11 and Annex IV, logging under Articles 12 and 19, human oversight under Article 14, cybersecurity under Article 15, deployer duties under Article 26, post-market monitoring under Article 72, and incident support under Article 73.

Because “we assumed the model would behave” is not a strong compliance strategy.

‍