Reference · Case AI Interview

Glossary

Terms adhered to across all lessons. Grows one lesson at a time — currently through Lesson 0007.

Internet plumbing

DNS (Domain Name System): The internet's phone book: translates domain names (caseai.com) into IP addresses computers route by. Route 53 = AWS's DNS service.
DNS record: One entry in that phone book. A record: name → IP address. CNAME: name → another name (e.g. → a CloudFront distribution).
CDN (Content Delivery Network): A fleet of cache servers in hundreds of cities serving your static content from near the user. Cloudflare is one; CloudFront is AWS's.
Edge server: The CDN cache server closest to a given user — the "edge" of the network. Cache hit = served locally; miss = fetched once from the origin.
Origin (server): The real source behind a CDN — your S3 bucket or ALB/app. Edges shield it from most traffic.
TLS (Transport Layer Security): The encryption protocol behind HTTPS (the browser padlock; successor to SSL, whose name survives in "SSL certificate"). Encrypts data in motion between two machines so nothing in between can read or tamper with it.
HTTPS: Simply HTTP running over TLS. TLS itself is protocol-agnostic: the same encryption also wraps non-HTTP traffic, e.g. the Postgres wire protocol on the container → Aurora hop.
Hop: One machine-to-machine leg of a request's journey (browser → CloudFront → ALB → container → Aurora = four hops). TLS protects one leg at a time, so "TLS on every hop" means each leg is separately encrypted — as HTTPS on the HTTP legs, as Postgres-over-TLS on the database leg.

AWS — networking & identity

Region: A geographic cluster of AWS datacenters (e.g. ap-east-1 = Hong Kong). You choose one; data and services live there.
Availability Zone (AZ): One of 3+ physically isolated datacenters inside a region. Deploying across AZs ("multi-AZ") survives a datacenter failure.
VPC (Virtual Private Cloud): Your own private network inside AWS. Everything server-side (containers, databases, load balancers) lives inside one.
Subnet (public / private): A slice of the VPC. Public subnets are internet-reachable (ALB, NAT); private subnets are not — app services and databases go there.
Security group: A stateful, per-resource firewall: rules like "allow the ALB to reach the app on port 443, the app to reach the DB on 5432, nothing else."
NAT Gateway: Lets private-subnet workloads make outbound internet calls (e.g. to the Anthropic API) while staying unreachable inbound.
IAM (Identity and Access Management): AWS's permission system for humans and workloads. Governs every API call to every service.
IAM policy: A JSON document listing allowed/denied actions on specific resources ("s3:GetObject on bucket X").
IAM role: An identity that a workload (an ECS task, a Lambda) assumes to receive short-lived credentials. The mechanism behind "no hardcoded keys."
Least privilege: Grant only the specific permissions a task needs, nothing more. The first phrase to say in any IAM answer.

AWS — compute, data, messaging

ALB (Application Load Balancer): The HTTP(S) front door of the VPC: TLS termination, health checks, routing traffic across containers in multiple AZs.
EC2 (Elastic Compute Cloud): The original AWS product: a raw rented virtual machine — pick CPU/RAM, get an OS, SSH in, patch it yourself, runs 24/7. Everything else (ECS, Fargate, Lambda) is an abstraction layer above it.
ECS (Elastic Container Service): AWS's container orchestrator (the simpler alternative to Kubernetes/EKS). You declare a task definition (image, CPU/memory, IAM role) and a service ("keep 3 copies behind the ALB"); ECS schedules, heals, and scales them.
Fargate: Serverless capacity for ECS — you declare CPU/memory per task and never manage servers. For long-running services.
API Gateway: Managed HTTP front door for Lambda-based backends: routing, auth, throttling — the "ALB of the fully-serverless stack."
Lambda: Event-driven functions: scale-to-zero, per-invocation billing, cold starts, 15-minute maximum runtime.
Cold start: Latency on a Lambda's first invocation after idleness, while AWS provisions the sandbox.
RDS / Aurora: Managed relational databases (PostgreSQL/MySQL). Aurora is AWS's cloud-native flavour: faster failover, storage that auto-grows, read replicas.
DynamoDB: Serverless key-value/document database, single-digit-millisecond reads at any scale. The KV analogue that's a real primary database.
S3 (Simple Storage Service): Object storage — the AWS twin of R2 (R2 copies S3's API). Where Case AI's legal documents would live.
CloudFront: AWS's CDN — an explicit distribution you place in front of an origin (unlike Cloudflare, where the CDN is implicit).
Route 53: AWS's DNS service.
SQS (Simple Queue Service): Message queue that decouples producers from consumers; retries, visibility timeouts, dead-letter queues.
Dead-letter queue (DLQ): A holding queue for messages that keep failing processing — inspect and replay instead of losing work. Strong reliability talking point.
SNS (Simple Notification Service): Pub-sub fan-out: one event, many subscribers (queues, Lambdas, emails).
Secrets Manager: Managed store for credentials (DB passwords, API keys) with rotation; apps fetch secrets at runtime via IAM instead of baking them in.
CloudWatch: AWS's built-in logs, metrics, and alarms — the default observability answer.
Well-Architected Framework: AWS's design doctrine: six pillars — operational excellence, security, reliability, performance efficiency, cost optimization, sustainability.

Security & compliance

KMS (Key Management Service): AWS's key vault: HSM-backed encryption keys nobody (including AWS) can extract; services encrypt S3/Aurora/SQS data under them via envelope encryption; every key use is logged to CloudTrail.
Envelope encryption: Data is encrypted with a data key; the data key is stored wrapped (encrypted under the KMS key) next to the ciphertext. To read, the service calls kms:Decrypt — KMS checks key policy + IAM, unwraps the data key inside its HSMs, and returns it transiently. The KMS key itself never leaves KMS; disable it and every unwrap fails forever (the cryptographic kill switch).
CloudTrail: The AWS-level audit log: every API call with identity, source IP, and time. Distinct from CloudWatch (observability) and from your own app-level audit table (business actions).
Tenant isolation (silo / pool / bridge): SaaS Lens taxonomy: silo = dedicated resources per tenant; pool = shared resources with per-row enforcement; bridge = mix per component. Legal SaaS usually answers "bridge."
Row-level security (RLS): Postgres feature enforcing per-row access policies in the database itself, so a pooled query physically cannot return another tenant's rows.
OWASP Top 10: OWASP = Open Worldwide Application Security Project (renamed from "Open Web…" in 2023), a nonprofit security foundation. Its Top 10 is the canonical ranked list of web-app risks; current edition 2025, A01 = Broken Access Control. Companion list exists for LLM apps (prompt injection etc.).
Prompt injection: Hostile instructions embedded in content the LLM processes (e.g. inside an uploaded contract) that hijack its behaviour — the injection risk of the AI era.
SOC 2 / Trust Services Criteria: SOC = System and Organization Controls, the AICPA's report suite (SOC 1 = financial-reporting controls; SOC 2 = trust/security controls at service organizations; SOC 3 = public summary). SOC 2 is an auditor's attestation that controls meet the Trust Services Criteria (Security mandatory; Availability, Processing Integrity, Confidentiality, Privacy optional). Type I = design at a point in time; Type II = operated over months.

AI terms met so far

RAG (Retrieval-Augmented Generation): Fetch relevant document chunks first, then have the LLM answer from them — the architecture behind Case AI's document features. Detailed in Lesson 0003.
Embedding: A vector representing a text's meaning; similar texts get nearby vectors, which is what makes semantic retrieval searchable.
pgvector: Postgres extension adding vector types + similarity search, so embeddings live next to relational case data in Aurora/RDS — under the same row-level security. Exact search (perfect recall) or approximate via HNSW indexes (faster, slight recall trade-off).
Hallucination: The model asserting things not grounded in evidence — invented clauses, made-up citations. RAG's answer-only-from-chunks + programmatic citation checks are the containment.
Chunk / chunking: Splitting documents into small retrieval units (a clause, a section). Structure-aware splitting beats fixed-size for contracts. Chunks lose document context when isolated — see contextual retrieval.
Contextual retrieval: Anthropic's technique: an LLM prepends a short situating sentence to each chunk before embedding/indexing ("from the 2024 Acme MSA, §12…"). ~49% fewer failed retrievals; ~67% with reranking.
BM25: The classic keyword-ranking algorithm — exact-token matching. Catches what embeddings miss: party names, "Section 12.3", statute numbers.
Hybrid search: Running semantic (embedding) and keyword (BM25) retrieval together and fusing the rankings. The production default; pure-vector search is a junior tell.
Reranking: A second, more careful model re-scores the top retrieved candidates so the best few chunks enter the prompt. Cheap accuracy gain at the end of the pipeline.
Top-k: How many chunks retrieval returns to the prompt. Too few = missing evidence; too many = noise and token cost.
Textract: AWS's ML-OCR service: extracts text, handwriting, tables, and key-value pairs from PDFs/scans with structure preserved — the extraction stage of the pipeline.
Citation grounding: Chunks enter the prompt with IDs; the model must cite IDs for every claim; the app resolves IDs to document + page and programmatically verifies the quote. Measurable → testable → CI.
Response normalization: One internal response shape; an adapter per provider maps vendor formats (finish reasons, token counts, tool calls, errors) into it. The payment-gateway pattern applied to LLMs.
Provider routing / fallback: Policy choosing which model serves which task (fit, cost, health); on failure: retry with backoff → circuit breaker → next provider in the chain, with degraded responses labeled, never silent.
Structured outputs: Generation constrained to a supplied JSON Schema (OpenAI native; via tool definitions with Claude). Guarantees shape, not truth — validate on receipt, retry on mismatch.
Tool calling (function calling): The model requests function invocations; your app validates the arguments (untrusted input!) and executes them tenant-scoped, feeding results back. The mechanical core of "multi-step AI workflows."
Workflow vs agent: Anthropic's taxonomy: workflows = LLM calls orchestrated through predefined code paths (chaining, routing, orchestrator-workers, evaluator-optimizer); agents = the model directs its own next step. Legal default: workflows.
HITL (human-in-the-loop): Every AI output is a proposal in a state machine (draft → review → approved/edited → released); a lawyer approves before release; edits become eval labels; approvals hit the audit table.
Golden set: A versioned, curated exam: real anonymized questions with known answers + supporting evidence, including edge-case traps (not-in-the-docs, jurisdiction, injection). Grown from production corrections.
LLM-as-judge: A model grading outputs against a written rubric for what code can't score (tone, completeness). Trustworthy when given examples, made to reason before scoring, and calibrated against human labels.
recall@k: Retrieval-layer metric: how often the labeled relevant chunk appears in the top k results. Code-graded, LLM-free, runs in seconds.
Eval / regression gate: The test suite for non-deterministic dependencies: golden set + graders run in CI on every prompt/model/pipeline change; score drops below threshold block the deploy.