Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Introduction to LLMOps
- Comparing LLMOps and MLOps: distinct challenges in operating LLMs
- The LLM application lifecycle: prompt engineering, evaluation, deployment, and monitoring
- Checklist for ensuring production readiness of GenAI applications
Prompt Management and Versioning
- Prompt templating systems and the use of variable injection
- Applying semantic versioning to prompts using automated regression testing
- Prompt registries and collaborative workflows
LLM Evaluation at Scale
- Key evaluation dimensions: accuracy, relevance, safety, and groundedness
- Using LLMs as judges and establishing human evaluation pipelines
- Automated evaluation frameworks including RAGAS, DeepEval, and custom evaluators
- Incorporating quality gates into CI/CD pipelines for LLM deployments
Safety Guardrails and Content Governance
- Implementing input and output guardrails with NeMo Guardrails and Guardrails AI
- Detecting PII, filtering toxicity, and defining topic boundaries
- Defending against jailbreaks and prompt injection attacks
- Conducting red-teaming exercises to ensure LLM application safety
LLM Observability and Monitoring
- Collecting telemetry for token usage, latency, cost, and quality metrics
- Detecting drift in LLM outputs and embedding spaces
- Implementing session-level tracing for multi-turn agent conversations
- Creating dashboards and setting up alerts using LangSmith, Arize, and OpenTelemetry
AI Gateway and Model Orchestration
- Multi-provider routing using LiteLLM and Portkey
- Configuring fallback strategies, retry logic, and circuit breakers
- Selecting models based on cost and implementing load balancing
- Managing rate limits, quotas, and API key governance
Performance Optimization
- Implementing semantic caching with vector stores and exact-match strategies
- Enforcing structured output via constrained decoding
- Applying batching, streaming, and concurrency patterns
- Optimizing latency across different model providers
Governance, Compliance, and Audit
- Establishing LLM audit trails: prompt logs, response logs, and decision provenance
- Considering data residency and privacy requirements for LLM APIs
- Implementing policy-as-code for LLM usage within organizations
- Developing an internal LLM operations playbook
Requirements
- Practical experience in building or integrating LLM-powered applications.
- Proficiency with Python and REST APIs.
- Fundamental understanding of prompt engineering principles.
Target Audience
- ML engineers and MLOps practitioners shifting focus toward LLM operations.
- Platform engineers overseeing LLM infrastructure.
- Technical leads managing production-level GenAI deployments.
14 Hours
Testimonials (2)
use of proper and effective prompt
Marses Pacaldo
Course - Generative AI and Prompt Engineering for Corporate Professionals
The interactive style, the exercises
