WA3551
Foundations of Responsible AI with the NIST AI Risk Management Framework Training
This Responsible AI course is about not just theoretical knowledge but also its practical application in real-world scenarios. It teaches the knowledge and skills needed to responsibly design, develop, and deploy artificial intelligence (AI) systems.
The training investigates the NIST (US National Institute of Standards and Technology) AI Risk Management Framework (AI RMF), the premier AI guidelines for US-based private and public sector companies. Participants gain practical experience applying the AI RMF to real-world scenarios through interactive discussions, case studies, and hands-on exercises.
Course Details
Duration
2 days
Prerequisites
No prior experience is required.
Skills Gained
- Understand potential harms posed by AI systems
- Describe the challenges of AI Risk Management
- Learn the Lifecycle and Key Dimensions of AI systems
- Discover the 7 characteristics of Trustworthy and Responsible AI
- Explore NIST AI Risk Management Framework’s four functions: Govern, Map, Measure, and Manage
- Apply the NIST AI RMF to tailored real-world case studies
- Develop strategies to avoid, reduce, transfer, diversify, and mitigate AI risk
- Explore potential legislation and the future of Responsible AI
Course Outline
- Introduction to Risk Management
- What is Risk?
- What is Risk Management?
- Why a Risk Management Framework?
- The NIST Risk Management Framework (RMF)
- Frame, Assess, Respond, Monitor
- Risk Management Roles
- Prepare
- Categorize
- Select
- Implement
- Assess
- Authorize
- Monitor
- Legacy of the NIST RMF
- Case Study: Risky Business
- Introduction to AI and Its Societal Impact
- What is Artificial Intelligence?
- Defining terms
- Brief history and overview of AI technologies
- Understanding AI’s Potential Benefits
- Understanding AI’s Potential Harms
- The Need for Responsible AI
- Introduction to AI Systems
- The AI System Lifecycle
- Common AI Actors
- Assigning activities to AI lifecycle phases
- Understanding AI Risks and the NIST AI Risk Management Framework (RMF)
- What is AI Risk?
- AI Risks vs. Traditional Software Risks
- Challenges in AI Risk Management
- Introduction to the NIST AI RMF
- Attributes of the NIST AI RMF
- Alternatives to the NIST AI RMF
- Core Functions: Govern, map, measure, manage
- Is it an AI risk?
- Understanding Trustworthy AI with Case Studies
- Safe
- Secure & Resilient
- Explainable & Interpretable
- Privacy-Enhanced
- Fair
- Accountable & Transparent
- Valid & Reliable
- Issue Spotting: Violations of Trustworthy AI
- Govern
- Policies, processes, procedures, and practices
- Accountability structures
- Diverse Input
- Culture & Communication
- Engagement Processes
- Policies and Procedures
- Contingency Plans
- Issue Spotting: Governance Structures
- Map
- AI in Context: Application
- AI in Context: Actors
- AI in Context: Aspirations
- Risk Assessment Process
- Categorization of AI
- AI Limitations and oversight
- Cost—Benefit Analysis
- AI System Scoping
- AI Operator Training
- Lab – Finding Risks in Sample Applications
- Measure
- Fit-for-Purpose
- Testing procedures
- Acceptable limits
- Categories of metrics
- Data lineage
- Documenting errors
- Evaluating TEVV processes
- Perverse incentives
- Independent testing & red teaming
- Production monitoring & safety statistics
- Chaos Engineering
- Labeling hazards
- Fairness and bias—in all forms
- Environmental impacts
- Comparison tests
- Emergent risks
- Frontier risks
- Measuring the metrics
- Feedback loops
- Gas station bathrooms & effective communication
- Risk Report Design
- Manage
- Go / No Go – When to develop and use AI
- AI Risk Prioritization
- Mitigation
- Transfer
- Avoidance
- Acceptance
- Insurance
- Sustaining value
- Emergency Stop
- Supply-chain Risks
- Pre-trained models
- Sunset clauses
- Continual improvement
- Incident response
- Transfer/Avoid/Mitigate/Accept
- AI RMF: Generative AI Profile
- CBRN Information
- Hallucinations
- Dangerous output
- Data privacy
- Environmental impact
- Human-AI Configuration
- Information Integrity
- Cybersecurity
- Intellectual Property / Controlled Information
- Abusive Content
- Toxicity, Bias, and Homogenization
- Supply Chain
- Evaluating GenAI Risk
- Capstone
- End-to-End AI Risk Management Case Study
