Lesson 0005 · ~20 minutes
Case AI calls multiple model providers for three reasons: fit (a cheap fast model for classification and chunk-context generation, a frontier model for drafting), cost (routing 80% of calls to models 10–20× cheaper is real money), and resilience (a provider outage must not be a product outage). The pattern you already know from payment gateways applies verbatim: define one internal interface — complete(request): NormalizedResponse — with an adapter per provider. Response normalization means every adapter maps its provider's shapes (finish reasons, token counts, tool-call formats, error types) into your one internal shape, so the rest of the codebase never knows which vendor answered.
Routing is a policy on top of that interface: by task type ("summarize" → cheap tier; "draft response to opposing counsel" → frontier tier), by tenant plan, or by current provider health. Fallback handling is what happens when a call fails: retry with exponential backoff on transient errors (429/5xx), then fail over to the next provider in the chain — and mark degraded responses as such rather than silently swapping quality. Two backend classics to name: timeouts (never wait unboundedly on a model) and a circuit breaker (stop hammering a provider that's down; probe it periodically). Streaming (token-by-token to the portal) is the other production reality — one more reason the API tier is a long-lived Fargate service, tying back to Lesson 0002.
Extracting parties, dates, and obligations from a contract must yield machine-usable JSON, not an essay. Structured outputs constrain generation to a JSON Schema you supply — OpenAI enforces schema adherence natively (docs); with Claude the same is achieved via tool definitions (tool use docs). Production rule regardless of provider: validate against the schema on receipt, retry on mismatch — the validation layer is yours, not the vendor's. (You met this pattern in Lesson 0004: chunk-context generation and citation IDs are structured outputs too.)
Tool calling (= function calling) inverts the flow: you describe functions to the model (search_cases(query), get_document(id)), and the model responds with a request to invoke one; your code executes it and feeds the result back; the model continues. That loop is what "multi-step AI workflows" means mechanically (OpenAI guide, Claude guide).
Anthropic's taxonomy (Building Effective Agents) is the shared vocabulary: workflows are LLM calls orchestrated through your predefined code paths (prompt chaining, routing, orchestrator-workers, evaluator-optimizer); agents are loops where the model itself directs its next step. Legal work wants predictability, so the senior answer is: workflows by default, agentic loops only where open-endedness earns its risk — research, not filing.
Human-in-the-loop (HITL) is the non-negotiable for legal: every AI output is a proposal, persisted with a status (draft → under review → approved/edited → released), and a lawyer approves before anything leaves the system. Design implication worth saying aloud: HITL is not a popup — it's a state machine in your schema, with the review queue as a first-class feature, edit-diffs captured (they become eval labels — next lesson), and the audit table from Lesson 0003 recording who approved what, when.
Read Anthropic's "Building Effective Agents" (~20 min). Its workflow taxonomy (prompt chaining, routing, orchestrator-workers, evaluator-optimizer) is the exact vocabulary for the "agent workflows" line of the job post — and its closing advice (simplest pattern that works) is the senior posture for this interview.