The short answer
AI Red Teaming may include two commands: Use AI agents in authorized security investigations, and investigate the safety of your own AI applications. State in advance which of the two you mean.
AI agents as part of security investigation
Agents can support research by bringing together information and investigating agreed scenarios. The investigator determines the limits, assesses the findings and remains responsible for the test. Automatically generated results must be validated before you base decisions on them.
Challenge your own AI application
An assistant may have access to documents or business software. Then the question is not only whether it gives a good answer, but also whether it discloses information or performs actions that are not intended. Test both normal user questions and misleading commands and unexpected source content.
Make a distinction between an error response and an incident
A clumsy formulation is something other than an unauthorized action. Review findings on the data involved, permissions and possible business impact. Keep track of which version of the application and which settings have been tested so that results remain comparable later.
From finding to secure configuration
Improvements may include access rights, data separation, controls on actions or human approval. Test the updated application again. JViT combines this research with guidance for safe AI implementations.
Discuss this with your team
- Are we testing with AI or are we testing our AI?
- What data and actions can the agent achieve?
- Who validates the findings?
- How do we check the improvements?
