GenAI Engineering

Production-oriented guidance for engineers building backend systems that use large language models. This track focuses on reliable software and measurable system behavior—not prompt tricks or model demos alone.

Scope and boundaries

GenAI is an independent track that complements the handbook's backend and systems core. Topics should connect model capabilities to APIs, data pipelines, security, observability, evaluation, cost, and operational reliability.

Learning path

Stage	Topics	What to understand
1. LLM foundations	Tokens, context windows, inference, sampling, structured outputs	Model behavior, limits, latency and output variability
2. Model integration	Provider APIs, streaming, timeouts, retries, fallbacks	Safe integration patterns in backend services
3. Embeddings and retrieval	Embeddings, chunking, indexing, vector search, hybrid search, reranking	Retrieval quality and relevance trade-offs
4. RAG systems	Ingestion, retrieval, grounding, citations, freshness	End-to-end retrieval-augmented generation
5. Agentic workflows	Tool calling, state, planning, LangGraph, workflow orchestration	Bounded, observable multi-step execution
6. Evaluation	Test datasets, retrieval metrics, answer quality, regression tests	Repeatable measurement instead of subjective demos
7. Production operations	Tracing, latency, token usage, caching, budgets, rate limits	Operating cost-effective and reliable AI features
8. Security and governance	Prompt injection, data leakage, authorization, PII, tenant isolation	Threat modeling and defense in depth

Topic index

Foundations

• Tokens, context windows, and inference
• Sampling parameters and deterministic behavior
• Structured outputs and schema validation

Backend integration

• Provider abstraction and model routing
• Streaming responses and cancellation
• Timeouts, retries, circuit breakers, and fallbacks
• Async execution, queues, and background jobs

Retrieval and RAG

• Embedding models and vector similarity
• Document parsing, chunking, and metadata
• Vector databases and index design
• Hybrid retrieval and reranking
• RAG evaluation, citations, and freshness

Agents and workflows

• Tool/function calling
• Stateful workflows and LangGraph
• Human approval and bounded autonomy
• Idempotency, retries, and durable execution

Production quality

• Offline and online evaluation
• Observability and tracing
• Prompt/model versioning and regression testing
• Cost, latency, caching, and rate limiting
• Prompt injection and data access controls

Standards for every guide

Each substantial page should explain the problem being solved, architecture and data flow, implementation choices, failure modes, security implications, operational metrics, and trade-offs. Include runnable examples where practical and distinguish provider-specific behavior from general concepts.

For RAG and agentic systems, evaluate components independently as well as end-to-end. Do not claim that a vector database, framework, or larger model automatically improves answer quality.

Engineering principles

• Treat model output as untrusted input; validate it before use.
• Enforce authorization in application and data-access layers, not in prompts.
• Keep tools narrowly scoped and validate every tool argument.
• Make workflows observable, bounded, cancellable, and safe to retry.
• Measure retrieval and answer quality with representative test sets.
• Track latency and cost per successful task, not only per model call.