Black-box versus white-box security tasks for AI, in plain words

Security testers describe their work by how much they know about the target. In white-box testing the tester starts with full knowledge: the source code, the design and often the credentials. In black-box testing the tester starts with almost nothing, usually just an address, and has to work out the rest. Testers also use the term gray-box for something in between. The same split applies when a model is the tester, and it changes what an environment teaches.

The difference in one table

	White-box	Black-box
What the model gets	The code, the design, sometimes the tests	An address or a shell, nothing else
The hard part	Reading a lot of code and spotting the flaw	Discovering what exists and how it works
Closest to	A code review or an audit	A real attacker approaching from outside
Typical goal	Find and fix the flaw	Get in and reach a stated goal
How it is checked	Tests, or a reference fix	Proof that the goal was reached

Security testing guides, such as the Open Worldwide Application Security Project's Web Security Testing Guide (version 4.2, December 2020), describe testing approaches that differ in how much the tester knows in advance. The terms are older than AI, and security teams use them every day.

Why the difference matters for AI

A model given the source code can often find a flaw by reading, and many public datasets are built that way: here is a vulnerable function, find the problem. That is useful, but it tests code reading more than attack skill.

A model given only an address has to do what an attacker does. It explores, notices what the system reveals, tries something, learns from the result and goes deeper. This is a chain of steps, often called an exploit chain, where each step depends on the one before. A task like this teaches planning under uncertainty, which is harder to find in public data.

Labs want both:

• White-box tasks train a model to review and repair code, which is the defender's job.
• Black-box tasks measure and train what an attacker can do, which tells a lab how risky a model is.

How a black-box task is built so it can be checked

For a task to be a good environment, a program must be able to tell whether the model succeeded. Builders usually do three things:

1. Break the goal into steps. Each step is a small checkpoint, such as finding the entry point or reaching a protected page.
2. Make each step prove itself. The model must submit a value that only appears when the step was completed, and the value changes on every run.
3. Keep the checker out of sight. The model never sees the checker or its logs.

This is the same idea as the verifiable rewards that labs prefer to an AI judge.

What it means if you build or keep these

If your company builds security training or testing material, ask which kind you hold:

• Do you have targets that give the tester nothing at the start, or do they hand over the code?
• Does each step produce a value that changes between runs?
• Can you run the same target again with different data, so it cannot be memorized?
• Do you also hold a fix task, so one set trains both attack and defense?

Companies with black-box targets, a clear checker and a record of how hard each one is are in a stronger position than those with a list of flaws. Read what a cybersecurity company can license and our page on selling RL environments. The glossary explains black-box and white-box testing in two sentences.