How to Engineer a Multi-Agent System - Harness, Contracts, and Verification

A thirty-minute lightning lesson on what a multi-agent system is actually made of: not the prompts, but the harness of deterministic code around the model. A multi-agent system is not a group chat between personas. It is several agent loops plus explicit coordination, state and control: code that splits a task into roles, routes work between them, holds shared state, enforces limits, and checks results before anything downstream relies on them. That harness is the difference between a demo that works once and a system that works every time. The session takes one worked architecture apart, a forecasting system in which independent research agents answer a resolvable question, a deterministic step aggregates their forecasts, and a supervisor investigates disagreement rather than averaging it away. Three decisions carry most of the design. First, when a subtask earns its own agent: when it needs different context, tools or permissions, when its evidence path is genuinely distinct, when it has its own result and evaluation criterion, or when it can fail without corrupting the rest. A role prompt that says "optimist" or "skeptic" may diversify behavior, but it defines no interface and no test, so it is not an architectural boundary. Second, what crosses each boundary: an input contract that rejects ambiguous or stale work early, an output contract carrying the forecast, its rationale, its evidence IDs and its uncertainty, and shared state that records inputs, outputs, decisions, failures and unresolved conflicts so a run can be replayed and compared. Third, where judgment ends and control begins. The model decides which query would reduce uncertainty and what the evidence implies; deterministic code decides which tools are permitted, how many steps are allowed, what output shape is required, and what happens on timeout. Verification is the part a demo skips. Before an output is accepted: is the schema valid, can each important claim be traced to a source or tool result, were the time, permission and budget boundaries respected, are conflicts with shared state detected, and does the output satisfy the real acceptance test for that subtask? An agent reporting high confidence has supplied metadata, not verification. The same standard applies to a supervisor, which earns its authority by obtaining new evidence or applying a separate resolution rule, never by voting, and whose own output needs a contract and a check. The session then works through an executed trace where three agents come back at 25%, 62% and 65% on the same question, and shows what the supervisor did with that spread. It separates a trace, which explains one run, from monitoring, which detects operational change across many, and from scoring, which tests whether a design change helped. It closes with six questions that inspect any multi-agent design, applied to a workflow from another domain to show the checklist transfers. Stefan Jansen is the author of Machine Learning for Trading, whose companion code has passed 20,000 GitHub stars, and the founder of Applied AI, which builds production ML for investment teams. He has been a systematic investment partner since 2013 and has taught ML to more than 110,000 professionals through DataCamp and General Assembly. ------------------------------------------------------------ Engineering a Multi-Agent Forecasting System - live, hands-on workshop Saturday, September 26, 10:00 AM to 3:30 PM ET https://maven.com/stefan-jansen/agent... You inspect the prompts, tools, contracts, config snapshots and complete traces of a working system; change a prompt failure mode, a data source and one configuration knob; run a forecast, a controlled comparison and the full daily cycle; and keep the repository, the dashboard, the guided notebooks and your saved runs. The two trading courses, both syllabi in full, unit by unit: https://ml4trading.io/courses/ The book and its companion code: https://github.com/stefan-jansen/mach... CHAPTERS 00:00 How to engineer a multi-agent system 01:18 Inspect a system, not a prompt 03:08 An agent puts a model in an action loop 04:26 Forecasting makes each part observable 05:46 The forecaster running live 06:39 Aggregation, disagreement, and the supervisor 09:31 The harness turns agents into a system 11:33 The repository exposes the whole run 13:59 Create agents when isolation adds value 15:44 A role prompt is not a system boundary 17:35 Contracts and state preserve meaning 20:26 Put judgment in models, control in code 22:17 Verify artifacts, not confidence 23:28 A supervisor must add a new check 24:29 An executed trace reveals the behavior 26:00 Trace, monitoring, and evaluation differ 26:36 Six questions inspect any design 27:18 The checklist transfers beyond forecasts 28:54 The workshop completes the build 29:47 The next live workshop 31:26 Q&A: build from scratch, or use a framework? 35:27 Closing