Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Leveraging EXO_OFFLINE to block runtime internet connectivity.
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors.
- Validating model weight integrity via SHA-256 checksums and signed model cards.
- Operating EXO within air-gapped networks, independent of HuggingFace services.
Dashboard and API Access Control
- Setting up and configuring reverse proxies (such as nginx or Caddy) with TLS termination.
- Deploying role-based access control mechanisms for the EXO dashboard and REST API.
- Utilizing macOS Keychain or Linux pass to securely store credentials for API authentication.
- Limiting access to administrative endpoints to designated source IP ranges.
Cluster Isolation and Network Security
- Segmenting EXO clusters using EXO_LIBP2P_NAMESPACE and VLAN configurations.
- Configuring host firewalls (including macOS Application Firewall, iptables, and nftables) for EXO ports.
- Mitigating risks associated with unauthorized device discovery and rogue node injection.
- Encrypting libp2p traffic between nodes in environments where RDMA is unavailable.
Model Governance and Provenance
- Establishing an internal model registry that includes approved model lists and associated metadata.
- Applying tags and versioning to quantized weights (4-bit, 8-bit) in tandem with source checkpoints.
- Restricting loading capabilities to specific HuggingFace repositories or internal artifacts only.
- Maintaining documentation on model lineage, licensing terms, and acceptable use policies.
Audit Logging and Compliance
- Directing EXO logs to immutable audit trails, such as SIEM systems or WORM storage.
- Correlating API call logs with user identities and precise timestamps.
- Recording key events, including model instance creation, deletion, and inference requests.
- Generating periodic compliance reports suitable for internal and external auditors.
Threat Modeling and Incident Response
- Identifying potential threats such as data exfiltration via model outputs, prompt injection, and side-channel leaks.
- Implementing pipelines for prompt monitoring and content filtering.
- Developing incident response runbooks tailored for cluster compromise scenarios.
- Isolating affected nodes, preserving forensic data, and reconstructing clean environments.
Physical Security and Hardware Boundaries
- Protecting Thunderbolt ports against unauthorized RDMA cable connections.
- Utilizing secure enclaves and Apple Silicon hardware attestation where applicable.
- Managing physical access to clustered Macs and shared storage resources.
- Documenting hardware lifecycle management and decommissioning procedures.
Regulatory Considerations
- Aligning EXO deployments with GDPR, HIPAA, and SOC 2 standards.
- Ensuring data residency by maintaining inference processes on-premise.
- Documenting vendor supply-chain risks involving MLX, EXO, and model weights.
- Preparing for AI governance frameworks, including EU AI Act Article 53.
Requirements
- Practical experience with EXO or alternative local LLM runtimes
- A solid grasp of Unix filesystem permissions and network ACLs
- Knowledge of TLS/SSL certificate management and fundamental encryption concepts
Intended Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators responsible for managing sensitive data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
