The Atlas take.
Promptfoo is a strong evaluation candidate for developers who want reproducible AI security tests close to application changes. Its agent and MCP guides are particularly useful because they distinguish a safe final response from unsafe intermediate tool execution. Atlas would use it to preserve high-confidence failure traces and targeted regressions, not to declare an application secure from a large pass count. Open-source, enterprise and hosted generation capabilities have different operating and data-handling boundaries. Promptfoo supplies assessment; production authorization and runtime enforcement still require their own controls.
02 / DEFINE THE BOUNDARY
What you are actually evaluating.
Separate open-source testing from enterprise service capabilities and any OpenAI platform integration. An evaluation result does not enforce production tool policy.
Promptfoo is an open-source evaluation and adversarial-testing framework with separate enterprise capabilities. This profile focuses on security testing of real agent behavior and MCP boundaries. A useful evaluation produces repeatable test cases and tool evidence that can become regressions in the application’s delivery pipeline.
- Deployment surface
- Custom/cloud agents
- Integration point
- Test harness
- Availability
- Confirm availability
Documented scope; confirm release status, supported integrations and entitlement for the selected offering.
03 / THE CONTROL PATH
Where it sits. How it works.
Atlas architecture interpretation: configured plugins/strategies exercise the target, available traces and graders assess outcomes, and validated failures become versioned regression tests tied to the application release.
- 01
Application/MCP target and instrumentation
Pin the target version and emit the tool/context evidence needed to observe the intended security outcome.
- 02
Generation, execution and grading
Run selected attacks and assertions with explicitly configured providers and data paths.
- 03
Validated failure and regression
Reproduce the consequence, fix it and retain a targeted test alongside legitimate-task controls.
Local runner, configurable model dependencies
Target evaluation runs from the local environment and sends requests to the configured target. Generation and grading may run locally or through hosted inference, with different data movement for each stage.
Hosted features require separate review
The data-handling guide describes remote-only plugins and hosted generation or grading, plus cloud sync, sharing, and setup helpers. A local command or a remote-generation flag is not a guarantee of network isolation.
Open source, enterprise, and ownership
Promptfoo’s press page states it is part of OpenAI and continues as an open-source project. The data guide describes separate enterprise on-premises deployment for isolated operation; confirm which features and inference arrangements are included.
04 / WHAT EARNS ITS PLACE
Strengths, with the evidence attached.
Atlas analysis of documented capabilities; availability and behavior depend on the edition and environment.
Intermediate actions can be tested
The agent guide documents trajectory evidence and assertions that distinguish final-answer safety from actual tool behavior.
MCP assessment is documented
The MCP guide supplies a scoped workflow for testing tool-connected systems rather than only model text responses.
Inspect the documented capability notes
Evidence reviewed: Agent, MCP, and red-team configuration documentation, plus the detailed data-handling guide and ownership statement.
Configurable targets and adversarial generation
Red-team configuration separates the target, attack-generation plugins, delivery strategies, and application purpose. The purpose guides attack generation; target-provider configuration does not by itself configure the attacker or grader.
Agent behavior and tool evidence
The agent guide supports testing full applications or individual components with custom providers. OpenTelemetry traces can expose model calls, guardrail decisions, tool executions, shell actions, searches, and errors rather than judging only final text.
Three different MCP threat models
The MCP guide covers attacks through a trusted client, malicious-server interactions, and direct MCP tool invocation. Direct server tests examine API-style authentication and input validation; agent tests examine behavior under hostile tool descriptions or responses.
Replayable reports and regressions
The documented flow initializes configuration, generates and runs attacks, then reports results. MCP examples show CI/CD integration, making retained test cases and explicit acceptance assertions useful for recurring release checks.
05 / KNOW THE COMPROMISES
The tradeoffs to take seriously.
Architectural implications and procurement questions from Atlas; these are not independently measured product failures.
Deep evidence needs instrumentation
Provider-level spans alone may not reveal the complete agent tool chain; configure and verify relevant trajectory fields.
Local settings are not total network isolation
Data-handling guidance identifies remote-only plugins and explains that disabling remote generation does not disable every network path.
Additional coverage boundaries
- The test result reflects the selected attacks, target version, and grader criteria. Review false successes and failures manually and record configuration to make results reproducible.
- A model-only endpoint can miss actual side effects. Instrument tools and verify resource state when testing authorization, exfiltration, destructive actions, or chained workflows.
- Budget for attacker and grader model calls, target usage, trace storage, and enterprise features. Review every configured provider and hosted feature before using sensitive test context.
06 / FROM PURCHASE TO PRACTICE
Make it work in your environment.
A proposed implementation sequence. Confirm supported versions, permissions, and rollback with the current documentation.
- 01
Freeze target and test configuration
Record application/model versions, tool permissions, plugin selection and generation/grading providers.
- 02
Use synthetic observable actions
Create expendable resources and tracing that can prove authorization failures without exposing customer data.
- 03
Keep adjudicated regressions
Turn a confirmed failure into a narrow repeatable assertion, then run it with legitimate tasks after relevant changes.
07 / THE OPERATING COMMITMENT
The bill is only part of the cost.
Commercial shape
Compare open-source operation with enterprise functionality and hosting requirements. Include target-model, generation/grading, infrastructure and reviewer costs; ownership announcements do not establish every platform entitlement.
- Generation, target and grading model consumption
- Selected enterprise/hosting features and infrastructure
- Instrumentation, finding review and regression maintenance
08 / COMPARE THE JOB
Put these beside it.
Alternatives worth examining for the same problem. Check each offering’s scope before treating it as a substitute.
HiddenLayer AI Attack Simulation
Compare automated adversarial coverage, target integration and reproducible findings using the same application.
CiscoCisco AI Defense
Compare model/application validation and report evidence while separately evaluating any desired runtime enforcement.
09 / MAKE THE DECISION OBSERVABLE
Earn confidence through a real evaluation.
Suggested by Atlas. Agree acceptance criteria, use synthetic fixtures in an authorized environment, and capture both useful work and failed controls.
- TEST 01
Capture a synthetic unsafe intermediate tool action
Acceptance evidence: Verify the trace exposes the action even if the final response looks safe.
- TEST 02
Turn the failure into a trajectory regression
Acceptance evidence: Preserve tool name, relevant arguments and expected denied behavior.
- TEST 03
Audit actual assessment network paths
Acceptance evidence: Record providers, remote plugins, telemetry and any sanitized trace feedback.
START WITH ONE EXERCISE
An evaluation agent reads a synthetic document that asks it to disclose a planted secret.
Evidence to look for: Preserve the case, tool configuration and actual disclosure outcome before and after repair.
Questions to bring to the vendor
- Does this test exercise real tools or a model endpoint only?
- Which capabilities require the enterprise service?
- How are test data and evaluator model costs handled?
10 / SHOW THE WORK
Evidence, and what is still unknown.
What this research cannot establish
- Atlas has not independently benchmarked attack coverage or grader accuracy.
- Confirm current open-source/enterprise boundaries and exact provider/plugin data handling.
Dossier research reviewed October 18, 2026. Earlier profile references are retained for traceability; products and commercial terms change.
- Promptfoo: agent testing and trace evidence ↗Product information
- Promptfoo: MCP security-testing scenarios ↗Product information
- Promptfoo: targets, plugins, and strategies ↗Product information
- Promptfoo: generation, grading, and data handling ↗Product information
- Promptfoo: project and ownership context ↗Product information
KEEP BUILDING YOUR JUDGMENT