Testing AI Systems with DeepEval: AI Agents, Chatbots & RAG

Set up Anthropic (Claude) and OpenAI keys for AI agents by adding a .env file, then run and test the agent using tool-based QA evaluation.

Learn how the deep evals task completion metric measures end-to-end task completion for AI agents, using black-box testing with input-output evaluation and white-box insights into tool use.

Learn to read DeepEval result reports by integrating OpenAI and Confident.ai keys, run task completion evaluations with a GPT-4 judge, and interpret dashboarded test results.

Explore how to test AI systems with DeepEval, focusing on AI agents, chatbots, and RAG, and understand internal workflow through tracing components and evals iterator.

Merge multiple metrics into a single test file for end-to-end agent testing, focusing on answer relevancy, task completion, and prompt alignment. Use a 0.7 threshold with golden inputs and tracing.

Explore testing multi-turn conversations with a customer support chatbot, validating memory, context retention, and follow-up accuracy using tool calls and quality metrics in deep eval.

Explore multi-turn chatbot testing using a black-box approach, building a four-turn conversation with a turns list and a conversational test case to evaluate metrics like turn relevance and knowledge retention.

Show how to validate multi-conversational agents with the turn relevancy metric by evaluating each chat turn for relevance against the previous prompt in a DeepEval test.

Discover how to test chatbots with knowledge retention, memory of prior answers, and conversation completeness using history objects, production logs, and custom deep evals metrics.

Discover how rag-based llm architectures use retrieval, augmentation, and prompt engineering to query a vector database of proprietary data, enabling real-time, accurate answers.

Explore a demo of the RAG app used for testing and compare it with an AI agent demo, highlighting deep eval metrics like context precision and context recall.

Explore contextual precision as a metric for RAG output quality by checking if the top retrieved documents contain the answer. Use embeddings and vector databases with tracing against ground truth.

Validate rag agents using deep eval metric methods with two golden objects in the evaluation dataset, testing retrieval context and contextual precision against expected outputs.

Demonstrate data generation in DeepEval tests by creating a synthesizer, generating goldens from docs with a model, and evaluating bias with a dataset.

Demonstrates safety metric evaluation with deep eval, showing bias and toxicity thresholds below 0.5, PII leakage checks, and synthesized data generation to create goldens for testing AI agents.

Learn Python basics by creating a hello world program, printing output, and using variables and comments in PyCharm. Understand code indentation and simple multi-variable assignments.

Explore how Python handles data types at runtime, learn to print mixed data types using the format method, and use type checks to distinguish integers, floats, and strings.

Explore Python data types, including numeric types, strings, and lists. Learn list operations like indexing, slicing, insert, append, update, and delete, with dynamic typing and no explicit type declarations.

Learn how Python lists and tuples differ in mutability and syntax, and practice creating and printing dictionaries with key-value pairs and proper indentation.

Create an empty dictionary at runtime and load key-value pairs like first name, last name, and city, then read and print values from excel-driven data.

Explore python if-else logic, colon-based blocks, and indentation, with variables, printing, and comparison operators, to see how code runs.

Explore how Python for loops iterate over lists, print values, and compute multiples of two and the sum of the first five natural numbers using range and a summation variable.

Learn how for loops use ranges with start, end, and step, including default step of one; see examples of skipping initial indices and iterating over lists.

Master Python while loops by watching a counter reduce each iteration, and apply break and continue with if statements to control when the loop prints and ends.

Learn how to declare and call functions in Python using def, colons, and indentation. Pass parameters, print results, and return values with examples like greet me and add integers.

Explore python object oriented principles by building a calculator class with methods, variables, and a constructor; learn class declaration, indentation, object creation, and calling methods.

Understand how Python constructors are auto-called on object creation, including default and parameterized forms, and distinguish instance versus class variables with the self keyword.

Explore how python inheritance lets a child class inherit properties and methods from a parent calculator, access parent variables, and call the parent constructor when needed.

Master Python strings by learning indexing, slicing, concatenation, membership tests with in, and common methods like split and strip (including left and right strips), all through practical demos.