Course Outline
Introduction to AI Red Teaming
- Navigating the evolving AI threat landscape
- The critical role of red teams in AI security
- Key ethical and legal implications
Adversarial Machine Learning
- Attack vectors: evasion, poisoning, model extraction, and inference attacks
- Creation of adversarial examples using techniques like FGSM and PGD
- Differentiating targeted vs. untargeted attacks and defining success metrics
Evaluating Model Robustness
- Measuring robustness against various perturbations
- Identifying model blind spots and potential failure modes
- Stress testing classification, vision, and NLP models
Red Teaming AI Pipelines
- Mapping the attack surface across data, model, and deployment layers
- Exploiting insecure model APIs and endpoints
- Reverse engineering to understand model behavior and outputs
Simulation and Tooling
- Utilizing the Adversarial Robustness Toolbox (ART)
- Employing tools such as TextAttack and IBM ART for red teaming
- Leveraging sandboxing, monitoring, and observability tools
AI Red Team Strategy and Defense Collaboration
- Structuring red team exercises and defining clear objectives
- Effectively communicating findings to blue teams
- Embedding red teaming practices into broader AI risk management
Summary and Recommended Next Steps
Requirements
- A solid grasp of machine learning and deep learning architectures
- Proficiency in Python and major ML frameworks (e.g., TensorFlow, PyTorch)
- Knowledge of cybersecurity principles or offensive security methodologies
Target Audience
- Security researchers
- Offensive security teams
- AI assurance and red team specialists
Testimonials (3)
inventory and identifying the different risk exposures within AI
Gary Cook - Cybersecurity and Information Technology Risk Division
Course - Introduction to AI Trust, Risk, and Security Management (AI TRiSM)
I really enjoyed learning about AI attacks and the tools out there to begin practicing and actively using for security testing. I took a lot of knowledge away which I didn't have at the beginning and the course met what I hoped it would be. My favorite part shown from the training was Comet Browser and was amazed at what it could do. Definitely something will be looking into more. Overall it was a great course and enjoyed learning all OWASP GenAI Top 10.
Patrick Collins - Optum
Course - OWASP GenAI Security
The profesional knolage and the way how he presented it before us
