AI340: RAG System Implementer UPDATE

Agenda:

• Fundamentals & Architecture of a RAG System
  • RAG principle:
    • retrieval vs. generation,
    • typical failure modes (e.g., hallucinations, incorrect grounding)
  • RAG architecture building blocks: ingestion → index → retrieval → prompting → answering
  • Model / LLM usage:
    • selection criteria
    • context window
    • cost/latency trade-offs
    • answer formats
  • Prompt engineering for RAG:
    • roles
    • templates
    • citation/source attribution
    • answer formats
      
      
• Data Sources, Indexing & Retrieval Quality
  • Connecting data sources (e.g., documents, wikis, ticketing systems, policies) - concepts & patterns
  • Chunking/structuring, metadata, access concepts (incl. permissions)
  • Embeddings & vector search:
    • Core principles
    • Hybrid search / re-ranking (when it makes sense)
  • Data quality & compliance basics:
    • PII/data protection
    • Provenance
    • Versioning
    • Deletion/retention logic
      
      
• Implementation, Operations & Hardening
  • Implementing a prototype (end to end) incl. UI/chat integration or API service pattern
  • Evaluation:
    • Test sets/"golden set"
    • Offline checks
    • Quality metrics
    • Regression tests
  • Trustworthiness:
    • Source grounding
    • Confidence signals
    • “Don’t know” strategies
    • Knowledge validation
  • Security & governance:
    • Prompt injection risks
    • Content/policy checks
    • Logging/auditability
  • Extensions:
    • Agent workflows
    • Knowledge augmentation
    • Multimodal data
      
      
  • Practical component: building and iterating a RAG prototype, tailored to an example scenario, incl. final demo.

Objectives:

After the AI340 RAG System Implementer course, participants will be able to architect a RAG system in a clean and sound way (building blocks, data flow, responsibilities), robustly connect different data sources and make them indexable (chunking, metadata, versioning), systematically improve retrieval quality (e.g., re-ranking, filters, query optimization), design RAG prompts so that answers are consistent, source-based, and usable, establish an evaluation and operations approach (test sets, monitoring, regression), integrate security and governance guardrails (authorizations, logging, guardrails), and deliver a prototype as a foundation for piloting and production rollout.

Target audience:

The AI340 RAG System Implementer training is intended for:

• (AI) developers, ML/AI engineers
• Software architects / tech leads
• IT and compliance roles with a technical background (e.g., ISMS teams)
• Platform / MLOps roles who operate or integrate RAG systems

Prerequisites:

To be able to follow the pace and content of the AI340 RAG System Implementer training, the following prior knowledge is required:

• Basic knowledge of Python
• Basic understanding of LLMs
• For the exercises: development environment (notebook/IDE), basic toolchain (e.g., Git)

Description:

In this AI340 RAG System Implementer training, participants learn to design, build, and operate a Retrieval-Augmented Generation (RAG) system hands-on so that it delivers consistent, traceable, and “grounded” answers from their own knowledge sources. The focus is on an actionable end-to-end approach: from data sources and document pipelines through embeddings and vector search to prompt/response design, evaluation, security mechanisms, and integration into existing tools and processes.

The AI340 RAG System Implementer course is deliberately tool-agnostic: it teaches common architectural building blocks and proven implementation patterns that can be implemented with widely used frameworks and vector databases. A domain-relevant example (e.g., information security/ISMS) serves as the common thread to make clear how RAG systems can support work with extensive rule sets and internal policies—for example, when dealing with questions about ISO standards or current regulatory requirements (e.g., ISO 27001, DORA, NIS2, CRA). This is a technical enablement perspective, not legal advice.

The course concludes with a working prototype plus a set of checklists/blueprints to scale the approach cleanly within the organization (data access, roles, governance, monitoring).