GenerativeAI Professional Certification Program

Learning Outcomes:

• Explain the evolution from traditional ML to generative AI and foundation models
• Differentiate between discriminative and generative modeling paradigms
• Describe the Transformer architecture, self-attention mechanism, and positional encoding
• Identify key LLM families (GPT, LLaMA, Gemini, Claude, Mistral) and their trade-offs

Topics Covered:

• AI/ML landscape: supervised, unsupervised, and generative paradigms
• History & milestones: GANs → VAEs → Transformers → Foundation Models
• Transformer deep-dive: encoder-decoder, multi-head self-attention, positional encoding
• Pre-training objectives: causal LM, masked LM, seq2seq denoising
• Scaling laws, emergent abilities, and model benchmarks (MMLU, HumanEval, GPQA)
• Open-source vs proprietary LLMs: licensing landscape (Apache 2.0 vs community licenses vs proprietary APIs)

Hands-on Lab Activities:

• Lab 1: Explore tokenization with tiktoken — analyze token counts, vocabulary sizes, and encoding strategies across GPT-4o and open-source tokenizers; visualize token boundaries on code vs natural language
• Lab 2: Run inference on a pre-trained LLM (LLaMA 3 / Mistral 7B) via Hugging Face Transformers — compare outputs with varying temperature, top-p, top-k, and repetition penalty settings

Learning Outcomes:

• Design effective prompts using zero-shot, few-shot, and chain-of-thought techniques
• Apply structured prompt patterns (ReAct, persona, meta-prompting) for complex reasoning
• Implement context engineering strategies to manage what enters the LLM context window
• Evaluate prompt quality using systematic scoring frameworks

Topics Covered:

• Anatomy of a prompt: system instructions, user messages, assistant pre-fill
• Zero-shot, one-shot, and few-shot prompting strategies with example selection
• Chain-of-Thought (CoT), Tree-of-Thought, Self-Consistency, and Step-Back prompting
• Prompt patterns: persona, template, meta-prompting, ReAct reasoning
• Context engineering: managing retrieval context, tool outputs, memory, and system instructions within token budgets
• Prompt injection, jailbreaking risks, and defensive prompting techniques
• Prompt evaluation: relevance scoring, faithfulness checks, coherence metrics

Hands-on Lab Activities:

• Lab 1: Build a prompt engineering workbench using OpenAI API — systematically test and compare zero-shot vs CoT vs few-shot across code generation, mathematical reasoning, and summarization tasks; log results with scoring rubrics
• Lab 2: Implement a context engineering pipeline — given a 100K-token knowledge base, build a system that dynamically selects, prioritizes, and compresses context to fit within a 16K-token window while maximizing answer quality

Learning Outcomes:

• Integrate OpenAI, Anthropic, and open-source LLM APIs into Python applications
• Implement streaming responses, retry logic, rate-limit handling, and error recovery
• Build LLM-powered applications using LangChain and LlamaIndex frameworks
• Manage conversation context, memory types, and structured output parsing

Topics Covered:

• OpenAI Chat Completions API: models, parameters, JSON mode, structured outputs
• Anthropic Messages API: system prompts, tool use, extended thinking, prompt caching
• Hugging Face Inference API and local model serving with Ollama
• LangChain fundamentals: chains, prompt templates, output parsers, LCEL
• LlamaIndex basics: data connectors, node parsers, indexing, query engines
• Conversation memory: buffer, summary, token-window, and vector-store-backed memory
• Structured output extraction: JSON mode, Pydantic models, Instructor library

Hands-on Lab Activities:

• Lab 1: Build a multi-provider LLM gateway in Python that routes requests to OpenAI, Anthropic, or local Ollama based on task type — implement unified streaming, automatic retries with exponential backoff, and cost tracking per request
• Lab 2: Create a structured data extraction pipeline using Instructor + Pydantic — extract product specifications from unstructured e-commerce descriptions into validated, typed JSON schemas with error handling and retry logic

Learning Outcomes:

• Explain how text embeddings represent semantic meaning in high-dimensional vector space
• Select embedding models based on task requirements and benchmark performance
• Set up and query vector databases with metadata filtering
• Implement semantic search and hybrid retrieval pipelines (dense + sparse)

Topics Covered:

• Embedding evolution: Word2Vec → GloVe → BERT → Sentence-BERT → modern embeddings
• Models: OpenAI text-embedding-3, Cohere Embed v3, BGE, E5, Jina
• Evaluation: cosine similarity, MTEB, dimensionality and matryoshka embeddings
• Vector DB architecture: HNSW, IVF-PQ, metadata filtering, namespaces
• Chunking strategies: fixed, recursive, semantic, document-aware, parent-child
• Hybrid search: dense + BM25 with reciprocal rank fusion

Hands-on Lab Activities:

• Lab 1: Build a semantic code search engine — embed Python functions using OpenAI embeddings, store in ChromaDB with metadata (file, class, docstring), and retrieve relevant code by natural language queries with re-ranking
• Lab 2: Benchmark chunking strategies (fixed 512-token vs recursive vs semantic) on a technical PDF corpus — measure retrieval quality using hit-rate@k, MRR, and nDCG metrics across 50 test queries

Learning Outcomes:

• Architect end-to-end RAG pipelines for knowledge-grounded generation
• Implement advanced retrieval: re-ranking, query transformation, multi-step retrieval
• Evaluate RAG systems using RAGAS (faithfulness, relevance, context metrics)
• Optimize RAG for latency, accuracy, cost, and hallucination reduction

Topics Covered:

• RAG architecture: indexing → retrieval → augmentation → generation
• Naive vs Advanced RAG: failure modes and improvements
• Query transformation: HyDE, multi-query expansion, step-back prompting
• Re-ranking: Cohere Rerank, BGE Reranker, ColBERT
• Context strategies: stuffing, map-reduce, refine, tree-summarize
• Evaluation with RAGAS

Hands-on Lab Activities:

• Lab 1: Build an Advanced RAG pipeline using LlamaIndex — implement query decomposition, hybrid retrieval (vector + BM25 with reciprocal rank fusion), Cohere Rerank, and source citation over a multi-document technical knowledge base
• Lab 2: Evaluate and optimize the pipeline using RAGAS — auto-generate a synthetic test set, measure all four RAGAS metrics, iterate on chunking size, retrieval top-k, and reranker threshold to improve scores by at least 15%

Learning Outcomes:

• Explain limitations of vector-only RAG and when Graph RAG is a better fit
• Build knowledge graphs from unstructured text using LLM extraction
• Implement Graph RAG combining graph traversal with vector retrieval
• Query KGs using natural language via LLM-generated Cypher

Topics Covered:

• Vector RAG limitations: multi-hop, global summarization, entity resolution
• Knowledge graph fundamentals: entities, relations, triples, property graphs
• LLM KG construction: entities, relations, coreference
• Microsoft GraphRAG: community detection and hierarchical summarization
• Neo4j + LangChain integration and hybrid retrieval
• Use case selection: vector vs graph vs hybrid

Hands-on Lab Activities:

• Lab 1: Build a knowledge graph from a technical documentation corpus using LLM-based entity/relation extraction — store in Neo4j, visualize the graph, and implement natural language querying via LLM-generated Cypher
• Lab 2: Implement a hybrid Graph RAG system — combine Neo4j graph traversal for multi-hop entity questions with vector retrieval for general queries; compare answer accuracy against pure vector RAG on a 30-question benchmark

Learning Outcomes:

• Apply the decision framework: prompt vs RAG vs fine-tune
• Prepare datasets for instruction tuning
• Fine-tune an open-source LLM using QLoRA
• Evaluate and distill models for cost optimization

Topics Covered:

• Fine-tuning methods: LoRA, QLoRA, prefix tuning, adapters
• Dataset formatting, filtering, deduplication, synthetic data
• QLoRA training with PEFT/TRL and quantization
• Training config and evaluation strategies
• Model distillation and cost-performance trade-offs

Hands-on Lab Activities:

• Lab 1: Fine-tune LLaMA 3 (8B) using QLoRA on a custom instruction dataset for domain-specific code generation — configure 4-bit quantization, train with SFT Trainer, and merge LoRA adapters into the base model
• Lab 2: Implement model distillation — use GPT-4o to generate high-quality training data, fine-tune a smaller model (Mistral 7B) on the distilled dataset, and compare performance vs cost against the teacher model on a held-out benchmark

Learning Outcomes:

• Explain the RLHF pipeline from SFT through reward modeling to PPO optimization
• Implement Direct Preference Optimization (DPO) to align a fine-tuned model
• Understand Constitutional AI, RLAIF, and emerging self-alignment techniques
• Conduct systematic red-teaming to identify model safety failure modes

Topics Covered:

• The alignment problem: why pre-training and SFT alone are insufficient for safe deployment
• RLHF pipeline: SFT → reward model training → PPO optimization (conceptual deep-dive)
• Direct Preference Optimization (DPO): eliminating the reward model, loss function intuition
• Constitutional AI, RLAIF, and iterative self-alignment approaches
• Preference data collection: human annotation guidelines, inter-annotator agreement, synthetic preference generation
• Red-teaming methodology: systematic probing for harmful outputs, hallucinations, bias, and prompt injection vulnerabilities

Hands-on Lab Activities:

• Lab 1: Align a model with DPO using TRL — take the SFT model from Module 7, construct a preference dataset (chosen/rejected pairs for helpfulness and safety), train with DPO Trainer, and evaluate alignment improvements
• Lab 2: Red-team the aligned model — design a systematic evaluation covering toxicity, hallucination rates, prompt injection resistance, and refusal appropriateness; produce a structured safety scorecard

Learning Outcomes:

• Leverage code-specialized LLMs for generation, completion, review, and refactoring
• Integrate AI coding assistants (GitHub Copilot, Cursor, Claude Code) into development workflows
• Build custom code generation and automated testing tools using LLM APIs
• Assess and mitigate risks in AI-generated code: hallucinated APIs, license issues, security vulnerabilities

Topics Covered:

• Code LLMs: CodeLlama, StarCoder 2, DeepSeek-Coder V2 — architecture, training data, and benchmark performance
• AI coding assistants: GitHub Copilot, Cursor, Claude Code, Codeium — workflow integration patterns
• Code generation patterns: function synthesis from docstrings, test generation, refactoring, code translation
• Automated code review: bug detection, security vulnerability scanning, anti-pattern identification with LLMs
• Documentation generation: docstrings, README files, API documentation from source code
• Risks and limitations: hallucinated APIs, license compliance (copyleft contamination), security risks, over-reliance patterns

Hands-on Lab Activities:

• Lab 1: Build an AI-powered code review bot — ingest a Python repository, use an LLM to analyze each function for bugs, code smells, type issues, and security vulnerabilities; output structured review comments in GitHub PR format
• Lab 2: Create an automated test generation tool — given a Python module, generate pytest unit tests using an LLM, execute them, capture failures, feed errors back to the LLM for self-correction, and measure final code coverage

Learning Outcomes:

• Work with multimodal models that process and generate text, images, and audio
• Build applications using vision-language models (GPT-4o, Gemini, Claude Vision, LLaVA)
• Implement text-to-image generation and image understanding pipelines
• Integrate speech-to-text (Whisper) and text-to-speech capabilities into applications

Topics Covered:

• Multimodal architectures: vision encoders (ViT, SigLIP), cross-attention fusion, early vs late fusion strategies
• Vision-Language Models: GPT-4o, Gemini Pro Vision, Claude Vision, LLaVA — capabilities and API usage
• Text-to-image generation: Stable Diffusion, DALL-E 3, Flux — diffusion model fundamentals and controlability
• Image understanding: captioning, visual question answering, document/chart/diagram analysis with VLMs
• Audio models: Whisper (speech-to-text), ElevenLabs / XTTS (text-to-speech), audio understanding
• Emerging modalities: video understanding (Gemini, GPT-4o), video generation, and omni-modal models

Hands-on Lab Activities:

• Lab 1: Build a multimodal document analyzer — use GPT-4o Vision API to extract structured data (tables, charts, key figures) from scanned financial reports and engineering diagrams; output validated JSON with confidence scores
• Lab 2: Create an end-to-end voice-interactive assistant — integrate Whisper for speech input, an LLM for reasoning, and a TTS engine for spoken output; build a FastAPI endpoint supporting real-time audio streaming

Learning Outcomes:

• Design autonomous AI agents with planning, reasoning, memory, and tool-use capabilities
• Implement function calling and structured tool integration across OpenAI, Anthropic, and open-source APIs
• Build stateful multi-step agent workflows using LangGraph with conditional branching
• Integrate agents with external systems using Model Context Protocol (MCP) for standardized tool access

Topics Covered:

• AI agent architecture: perception → reasoning → planning → action → observation loop
• Function calling: OpenAI tools API, Anthropic tool use, parallel and sequential tool execution
• ReAct (Reasoning + Acting) agents: thought-action-observation cycles, scratchpad management
• LangChain agents: tool selection, agent executor, custom tool creation, structured tool outputs
• LangGraph: stateful workflows with conditional edges, cycles, persistence, and checkpointing
• Model Context Protocol (MCP): standardized tool integration, MCP servers, client architecture
• Human-in-the-loop patterns: approval gates, clarification requests, escalation, and fallback strategies

Hands-on Lab Activities:

• Lab 1: Build a ReAct research agent with custom tools — create an agent that can search the web, execute Python code, query a SQL database, and read files; orchestrate a multi-step research workflow that synthesizes findings into a structured report
• Lab 2: Implement a LangGraph customer support agent — design a stateful workflow with branching (ticket classification → intent routing → knowledge retrieval → resolution → escalation), persistence across sessions, and human approval gates at critical decision points

Learning Outcomes:

• Implement input validation, output filtering, and content moderation guardrails for LLM applications
• Deploy PII detection, hallucination checks, and toxicity filtering in production pipelines
• Use guardrail frameworks (NeMo Guardrails, Guardrails AI, LlamaGuard) to enforce safety policies
• Design responsible AI governance: bias auditing, fairness testing, and compliance with AI regulations

Topics Covered:

• Guardrail taxonomy: input guards (prompt injection detection, topic restriction, PII masking) vs output guards (hallucination detection, toxicity filtering, format validation)
• NeMo Guardrails: Colang flows, topical rails, fact-checking rails, moderation rails
• Guardrails AI (RAIL spec): validators, on-fail actions, structured output enforcement
• LlamaGuard & ShieldGemma: safety classification models for content moderation
• PII detection and redaction: regex + NER hybrid approaches, Microsoft Presidio integration
• Hallucination detection: self-consistency checks, NLI-based verification, source grounding validation
• Responsible AI: bias auditing, fairness metrics, EU AI Act basics, model cards, and transparency reporting

Hands-on Lab Activities:

• Lab 1: Build a multi-layered guardrail system using NeMo Guardrails — implement input rails (prompt injection detection, topic restriction, PII masking), output rails (hallucination check via NLI, toxicity scoring, format validation), and test against an adversarial prompt suite of 50+ attack vectors
• Lab 2: Integrate Guardrails AI into a RAG application — add Pydantic-based output validators, implement factual consistency checks against retrieved sources, build a PII redaction pipeline with Presidio, and generate a safety compliance report with pass/fail metrics

Learning Outcomes:

• Design comprehensive evaluation strategies: LLM-as-judge, automated benchmarks, and human evaluation
• Implement CI/CD-integrated prompt testing and regression detection pipelines
• Deploy LLM applications to production with optimized inference and observability
• Manage cost, latency, and reliability using caching, model routing, and fallback chains

Topics Covered:

• LLM evaluation paradigms: reference-based metrics (BLEU, ROUGE), LLM-as-judge, human evaluation protocols
• Automated testing: Promptfoo for prompt regression testing, DeepEval for unit-testing LLM outputs, CI/CD integration
• A/B testing prompts and models: experiment design, statistical significance, guardrail-aware rollout
• Serving frameworks: vLLM, TGI, Triton Inference Server — continuous batching, KV-cache, speculative decoding
• Deployment: cloud APIs (AWS Bedrock, Azure OpenAI, GCP Vertex), self-hosted, serverless architectures
• Inference optimization: quantization (GPTQ, AWQ), prompt caching (Anthropic, OpenAI), semantic response caching with Redis
• Observability: LangSmith, Langfuse, Phoenix — distributed tracing, cost tracking, latency monitoring, error dashboards
• Cost management: token budgeting, tiered model routing (complex → GPT-4o, simple → Haiku), fallback chains

Hands-on Lab Activities:

• Lab 1: Build an LLM testing and evaluation pipeline — use Promptfoo to create a test suite with 30+ test cases across accuracy, safety, and format dimensions; integrate with GitHub Actions for CI/CD prompt regression detection; implement LLM-as-judge scoring with calibrated rubrics
• Lab 2: Deploy a production-ready LLM application — serve a quantized model with vLLM behind FastAPI, instrument with Langfuse tracing, implement semantic caching with Redis, set up model routing (GPT-4o for complex queries, Haiku for simple ones), and build a Grafana-compatible monitoring dashboard tracking latency (TTFT, TPS), cost per query, and error rates

Learning Outcomes:

• Understand the fundamentals of LLMOps for managing LLMs in production
• Explore best practices and tools for deploying LLMs
• Design deployment pipelines for Generative AI applications
• Implement basic LLM deployment using containerization and orchestration
• Integrate n8n for automating deployment workflows

Topics Covered:

• Introduction to LLMOps: definition, importance, and lifecycle management of LLMs
• LLM Deployment Best Practices: model serving, API endpoints, latency optimization
• Containerization: Docker for packaging LLM applications
• Orchestration: introduction to Kubernetes for scalable deployment
• Model Serving Frameworks: TensorFlow Serving, TorchServe, Hugging Face Inference Endpoints
• CI/CD Pipelines for LLMs: automating build, test, and deployment
• n8n for workflow automation in AI deployments

Hands-on Lab Activities:

• Hands-on: Containerizing a simple LLM application using Docker
• Deploying a containerized LLM application to a local Kubernetes cluster (Minikube)
• Setting up a basic CI/CD pipeline for an LLM application (e.g., using GitHub Actions)
• Use n8n to create automated workflows for AI model deployments (e.g., automating model updates or monitoring tasks)

Learning Outcomes:

• Architect and build a production-grade Generative AI application integrating multiple modules
• Implement RAG (vector + graph), agentic tool use, guardrails, and observability in a single system
• Demonstrate systematic evaluation with automated test suites and safety benchmarks

Topics Covered:

• Project briefing: requirements specification, evaluation rubric, and deliverable checklist
• Architecture design session: component selection, data flow diagramming, integration planning
• Implementation sprint: guided build with mentor checkpoints at each integration stage
• Testing & evaluation: functional testing, RAG evaluation (RAGAS), safety checks (guardrail test suite), load testing
• Career pathways in Generative AI: roles (AI Engineer, LLM Ops, Prompt Engineer), certifications, and industry trends

Hands-on Lab Activities:

• Capstone: Build an AI-Powered Enterprise Knowledge Assistant — a full-stack application integrating: (1) Advanced RAG with hybrid vector + graph retrieval and re-ranking over a multi-format document corpus (2) Agentic tool use with LangGraph — code execution, web search, database queries, and MCP-based integrations (3) Multi-layered guardrails — input validation, PII detection, hallucination checking, and output filtering (4) Production infrastructure — FastAPI serving, Langfuse observability, semantic caching, and model routing
• Deliverables: working application, architecture document, Promptfoo test suite (30+ cases), RAGAS evaluation report, safety scorecard