Most production systems make decisions with one of two tools, and both eventually hurt. The first is a hand-written rule engine: an if/else lattice that starts readable and ends as a 4,000-line artifact nobody wants to touch, because every new edge case adds a branch and every branch multiplies the test surface. The second is a chat LLM wrapped in a “reply in strict JSON” prompt, which works in staging and then fails in production on malformed output, prompt drift, and round-trip latency measured in seconds rather than milliseconds. A decision engine is the third option — a service whose entire contract is: you send state, you get a typed, calibrated decision back, deterministically, fast enough to sit inline in a request path. This article breaks down what that means architecturally, how the API contract works, and where it earns its place in a real stack.
What a Decision Engine Is — and How It Differs From an LLM Endpoint
A decision engine is not a text generator with a schema bolted on. It is a model class purpose-built for the narrow task of mapping observed state to a bounded answer drawn from a fixed vocabulary you define. The output is typed: not a paragraph that happens to contain a label, but a value with a declared type and a declared range. In practice that means a handful of answer shapes cover almost every operational question — a choice (exactly one of your labels), a score (a level on an ordered scale), a truth (a calibrated probability between 0.00 and 1.00), a number (a value on a grid), and a point or box (a location inside an image).
The distinction matters most when you compare it to the “ask a chat model for JSON” pattern that many teams adopt first. That pattern carries three structural problems. First, format risk: the model may emit valid-looking JSON that violates your schema, and you end up building a repair layer around a system that was supposed to remove layers. Second, non-determinism: the same input can produce different answers across calls, which makes regression testing, caching, and incident replay difficult to reason about. Third, latency and cost shape: chat completions are priced and timed like document generation, not like a function call, so inline use in a request path becomes a budget conversation.
Hand-written rules fail in the opposite direction. They are perfectly deterministic and effectively free per evaluation, but they cannot absorb unstructured input — free-text tickets, images from a camera, a JSON payload with forty optional fields — without someone manually encoding every signal. The result is brittleness that compounds: rules encode what someone believed about the data at the time, not what the data actually looks like today.
A decision engine occupies the middle. It handles messy, multimodal state — text, structured JSON, and up to sixteen images or video frames — while still returning a bounded answer your code can branch on without parsing heuristics. That combination is what makes a decision engine usable as infrastructure rather than as a feature: the calling code does not need to know whether the answer came from a lookup table, a gradient-boosted model, or a neural network, only that the answer is typed and carries a confidence value.
Confidence is the part teams underestimate. A typed answer without calibration forces a binary trust decision — either you believe the model or you don’t. A calibrated confidence turns that into a threshold you can tune per call site, which is discussed in more detail below.
The Decision API Contract: State In, Typed Decisions Out
The integration surface of a decision engine should be boringly small, and in DecisionNode’s case it is: one HTTPS endpoint, POST /v1/decide. The request carries two things — state and questions — and the questions are defined in your codebase, not invented by the model at runtime. That inversion is important. You are not asking an open-ended system to figure out what matters; you are declaring the decision you need made, and the engine returns the value for it.
State can arrive as plain text, as JSON, or as up to sixteen images or video frames, which lets a single call reason across, for example, a support ticket body plus a screenshot plus a shipping manifest. Answers come back schema-valid in one pass, sized to the question type you declared:
A choice question returns one of your labels — routing a ticket to billing, infrastructure, or abuse; classifying a carrier exception. A score question returns a level on an ordered scale, useful for risk tiers or priority bands where the ordering itself carries meaning. A truth question returns a calibrated probability, which is the right shape for fraud signals, promo-abuse likelihood, or whether a photo depicts damage. A number question returns a value on a grid, suited to quantities and estimates. Point and box questions return coordinates, covering localization tasks like identifying a defect region or a package label within a frame.
Every answer arrives with a confidence value, and that is what makes the response safe to act on. The standard pattern is a threshold gate: above your cutoff, proceed automatically; below it, route to a re-check, a human, or a conservative fallback path. Because the engine is deterministic — the same request returns the same answer — those thresholds are testable. You can replay last month’s traffic against a new cutoff, assert on the distribution of outcomes, and ship the change with evidence rather than vibes.
Determinism also unlocks the usual operational niceties: cache the decision, replay an incident, diff two question definitions, and write assertions in CI that do not flake. SDKs exist for TypeScript, Python, Go, Ruby, and plain curl, with MCP tools and webhooks for teams wiring decisions into agent workflows or downstream queues. For AI agents specifically, the pattern is decide before acting: call the engine to validate an intended action, then execute only if confidence clears the bar.
Latency, Cost, and Throughput: Running a Decision Engine in Production
Inline decisions live or die on the p99. DecisionNode ships two models with different tradeoffs. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end, priced at $0.035 per 1M input tokens. DecisionNode MoE runs at roughly 9 ms server-side at $0.042 per 1M input tokens, for cases where the extra capacity pays for itself. In both cases output tokens are free, which changes cost modeling meaningfully: you are not paying per label returned or per probability emitted, so widening a question set does not multiply the bill the way it would with a generative endpoint.
That pricing shape matters for high-volume paths. Ticket triage, moderation queues, and order routing can each fire several questions per item, and the per-decision cost stays a rounding error rather than a line item. Batch discounts cover backfill and offline scoring runs, and a free daily tier lets teams validate the contract in staging before committing to volume.
Throughput is the other axis. Request/response covers most service traffic, but decision loops that exceed roughly ten decisions per second — drone flight control, robot manipulation, simulator stepping, continuous machine monitoring on a line — are better served by sessions over WebSocket, which hold context and stream decisions without per-call handshake overhead. Choosing between the two is a straightforward engineering call: stateless request/response for business workflow decisions, a persistent session when the decision rate is bounded by a control loop rather than by a user action.
There is also a metering question worth settling early. Since determinism guarantees repeatability, you can memoize decisions on a hash of state plus question set and cut call volume substantially for workloads with heavy duplication — the same product image re-evaluated across a batch, for instance. Combined with batch discounts, that means the honest capacity plan is often cheaper than the naive per-request math suggests.
Where Decision Engines Pay Off: Workloads, Boosters, and Real Scenarios
The recurring theme across successful deployments is that the decision is bounded but the input is not. If you can enumerate the inputs, a rule engine is fine. If you cannot enumerate them but you can enumerate the possible answers, a decision engine fits.
In support operations, a ticket-triage call combines the ticket body, structured metadata, and any attached screenshots, returning a routing choice plus a truth value for escalation likelihood. In fraud and promo abuse, the engine scores an order or signup against signals that rules handle poorly — mismatched device metadata, suspicious velocity, an image that looks synthetic — and returns a probability your code compares to a threshold. In order and shipping workflows, choice and number questions resolve exceptions: which fulfillment path, how many units to substitute, whether a photo of a delivered package supports a damage claim.
Visual inspection is where the typed-output model shines brightest, because box and point answers are unusable from a generative endpoint without a parsing layer. Quality control on a production line can ask for a defect box and a truth value for pass/fail, then branch on confidence — auto-reject above 0.9, flag for human review in the middle band. Warehouse monitoring asks whether a pallet is in the correct bay; traffic monitoring asks for vehicle counts on a grid. Boosters extend these calls with specialized capabilities: human age estimation for compliance gating, AI-generated image detection for authenticity checks, face detection and recognition for access scenarios, OCR for label and document text extraction, and depth or segmentation for spatial reasoning about a frame.
For product and platform teams, the most valuable framing is that a decision engine replaces a class of glue code. Every team eventually writes the same module — normalize input, apply heuristics, return a verdict with a confidence — and every team eventually regrets it. Moving that module behind a single endpoint with typed answers, calibrated confidence, and deterministic replay turns an accumulating liability into a versioned dependency. For ops and automation teams, the win is narrower but immediate: brittle hand-written rules become threshold-gated calls with a confidence signal attached, so exceptions surface explicitly instead of failing silently inside a branch nobody remembers writing.
Kinshasa blockchain dev sprinting through Brussels’ comic-book scene. Dee decodes DeFi yield farms, Belgian waffle physics, and Afrobeat guitar tablature. He jams with street musicians under art-nouveau arcades and codes smart contracts in tram rides.