Securing GenAI Systems

Explore how data analysis, machine learning, and deep learning enable generative AI to create content, with supervised, unsupervised, and reinforcement learning guiding neural networks.

Secure GenAI systems by managing dynamic, probabilistic behavior and training data as attack surface. Guard outputs with red teaming, guardrails, and AI risk governance to ensure trustworthiness.

Outline the high level architecture of gen ai systems, from data collection and tuning to embeddings, prompts, inference, and monitoring. Emphasize layered security to prevent data leakage and prompt injection.

Explore real-world Gen AI breaches and abuses, from memory leaks to prompt injections, hallucinations, and deepfakes. Secure the full stack with data loss prevention and human-in-the-loop checks.

Explore OWASP and its top ten for llm applications, including prompt injection, sensitive data disclosure, supply chain data and model poisoning, and output handling, to design secure gen AI systems.

Explore prompt injection in GenAI, its direct and indirect forms, including multimodal vectors, and why it risks data leakage, unauthorized actions, and safety breaches in GenAI systems.

Explore real-world prompt injection in direct and indirect forms, from system prompts being overridden to malicious data in websites and PDFs, and learn mitigation strategies.

Explore practical mitigation strategies against prompt injection, from system level controls to defensive prompt engineering and layered defenses.

Learn how sensitive information disclosure occurs when AI reveals data from training prompts or inputs. Understand why this risk matters for privacy laws like GDPR and HIPAA.

Examine real-world GenAI data leaks, from model memorization of training data to server-side memory bugs and unfiltered fine-tuning, including RAG use that exposes emails and API keys.

Explore mitigation strategies to protect sensitive data in generative AI, including training data de-duplication, redaction, governance, prompt filtering, and secure Rag deployments.

Explore supply chain vulnerabilities in generative AI from data, pre-trained models, libraries, and plugins, where poisoned weights, biases, and backdoors threaten security, integrity, and CI/CD pipelines.

Explore supply chain attacks in generative AI, including poisoned models, compromised libraries, poisoned data sources, and backdoored adapters, and learn how vigilance protects your deploys.

Verify model provenance and metadata with official sources and digital signatures, then enforce isolation, dependency scanning, and secure MLOps to harden the supply chain.

Explore data and model poisoning, including pre-training, fine-tuning, and embedding attacks, backdoors, and hidden triggers that persist across sessions and degrade AI integrity.

Explore real-world poisoning examples in GenAI systems, from backdoors triggered by a phrase to poisoned public data and biased vendor datasets.

Defend generative AI systems with robust data curation and validation, secure training pipelines, and monitoring. Employ canary releases, versioning, rollbacks, and audits to detect and stop model poisoning before production.

Understand how improper output handling creates security risk in generative AI by failing to validate and sanitize untrusted outputs before they reach users.

Explore real-world improper output handling in generative ai systems, illustrating log deletion, data leaks, phishing sites, cross-site scripting, sql injections, and content moderation failures to drive robust safety practices.

Apply mitigation strategies to secure GenAI systems by treating outputs as untrusted input, enforcing validation, sanitization, content filtering, and sandboxing with human-in-the-loop oversight.

Explore excessive agency in agentic AI, where too much autonomy and tool access enable harmful actions such as data modification, credential leaks, or IoT control, underscoring accountability and guardrails.

This lecture shows real-world scenarios where over-permissioned AI agents spiral out of control, illustrating the dangers of excessive agency and the need for surgically scoped permissions and human oversight.

Apply practical mitigation strategies for secure Agentic AI, including least privilege, tool whitelisting, human-in-the-loop control, RBAC, sandboxing, output validation, observability, and continuous red teaming.

Explore how system prompts govern AI behavior and how leakage via echoing, prompt injection, few-shot examples, and chain-of-thought can enable attackers to bypass guards and induce unintended responses.

Explore real-world system prompt leakage through documented exploits, from prompt dumps to jailbreaking and tool abuse, and learn practical mitigation strategies to secure genai systems.

Implement a defense strategy to prevent system prompt leakage by using system level configurations, wrapped APIs, security checkpoints, guardrails, red-teaming, and monitoring to block exposure.

Identify vector and embedding weaknesses in retrieval augmented generation systems and explore how embedding injection, inversion, and data poisoning threaten privacy and model behavior, with best practices and secure architecture.

Identify real-world weaknesses in vector and embedding systems. Examine embedding injection, poisoned vector stores, inversion attacks, and unauthorized access to embeddings in rag-based recommendations.

Defend the invisible backbone of retrieval augmented generation by applying eight actionable strategies to secure the vector store, sanitize inputs, obfuscate embeddings, filter results, and monitor anomalies.