Advanced LLM Application Engineering: Hybrid RAG, Tools, Memory & Evals

Extend a working AI agent with hybrid retrieval, tools, persistent memory, evaluation, and guardrails in a three-hour coding workshop.

**Workshop description**

Modify and test a prepared agent using Google ADK and Gemini. Connect document retrieval with analytical tools and simulated API actions, then investigate retrieval errors, stale memories, prompt injection, and tool failures.

Inspect traces and compare answer quality, task completion, latency, and model usage before and after your fixes.

**Who this workshop is for**

• Data scientists and ML practitioners developing agent-based applications.
• AI and software engineers working with retrieval, tools, and memory.
• Technical practitioners seeking practical evaluation and reliability controls.

**By the end, you will be able to build**

• Hybrid retrieval combining BM25, Chroma vector search, result fusion, filtering, and reranking.
• Analytical tools using pandas, SQLite, and simulated APIs.
• Persistent memory with scoped retrieval and conversation state.
• Tool controls with argument validation, approval checks, and bounded retries.
• Evaluation and debugging workflows using traces, structured logs, and pytest assertions.

**Prerequisites**

• Comfortable writing Python and working with APIs or data workflows.
• Familiarity with LLM applications is helpful. Google ADK experience is optional.
• Complete the provided environment and model-access checks beforehand. Setup instructions will explain access requirements and usage costs.

**Key takeaways**

• GitHub project with starter code, exercise checkpoints, and a completed reference implementation.
• Reusable retrieval, memory, tool, and guardrail components.
• Evaluation datasets, failure scenarios, scoring scripts, and sample traces.
• Architecture diagram, design cheatsheet, evaluation checklist, and setup guide.
• Full HD recording, presentation slides, and Certificate of Completion.

**About the instructor**

**Brij Kishore Pandey** is a Principal Engineer and AI Architect at Wells Fargo with more than 16 years of experience across AI, data engineering, cloud infrastructure, and enterprise software. His teaching focuses on building systems that can be evaluated, observed, and controlled under realistic conditions.