# A New Approach to AI Decision-Making: Structured Judgments Over Text Generation
## What if AI didn’t need to generate text at all?
There is a quiet but significant shift happening in how we think about AI systems for decision support. Rather than asking a large language model to write free-form responses and then parsing those responses with brittle heuristics, some teams are building systems where AI returns structured, typed judgments—labels, scores, and probabilities—that code can act on directly. This approach eliminates an entire layer of ambiguity: instead of hoping a model’s prose correctly encodes the information you need, you ask a precisely scoped question and receive a bounded answer.
One such system works with what its creators call “program state”—any structured or unstructured data you provide as context—and a dictionary of named questions. The model never generates a paragraph. It returns a set of decisions, each with a probability distribution over the possible answers you defined. This means the entire output is predictable in shape, easy to validate, and simple to compose into larger workflows.
## Three Fundamental Question Types
The core of this paradigm rests on three kinds of queries you can pose to the model:
**Selection (Choice):** You define a closed set of categories with descriptions, and the model picks the one that best fits the provided state. Each selection comes with a confidence score derived from the probability distribution across all options. This is useful for routing tasks—deciding which team handles a support ticket, classifying the intent of a user message, or categorizing an incident.
**Rating (Score):** You provide an ordered rubric of levels, and the model places the input state somewhere along that spectrum. Crucially, the answer can land between levels, weighted by the probabilities. For instance, if “low urgency” has a probability of 0.3 and “medium urgency” has a probability of 0.7, the expected score reflects that blend. This gives you a continuous measure rather than a crude bucket.
**Binary Verification (Noul):** You ask whether a statement is true, and the model returns the probability that it is. This is the tool for yes/no judgments—does the customer’s situation fall within the refund policy? Was customer data exposed during the incident? The output is already a probability, so there is no separate confidence field to track for this primitive.
All three types can be evaluated in a single request, each in isolation from the others, meaning one question’s answer does not influence another’s.
## Context Shapes Everything
The only thing the model knows is the state you provide. This makes the structure of your context critically important. Consider a simple question about eligibility for a refund. If you give the model only the customer’s message, it has one perspective. If you add the conversation history, it gains temporal context. If you also include the order details and the written refund policy, it can make a far more informed judgment. The most recommended approach is to pass structured data with named fields, so that instructions can reference specific parts of the context clearly and reliably.
## Measuring Certainty with Published Statistics
Confidence in this system is not an opaque metric. It is computed directly from the probability distribution the model returns. The published formula takes the number of possible answers, multiplies it by the highest probability among them, subtracts one, and divides by the number of answers minus one. This statistic reaches its maximum when the model is highly confident in a single answer and drops when the distribution is spread across multiple options. You can recompute it yourself from the probabilities and compare it against whatever confidence the system reports, giving you full transparency into how certain the model is.
猜想 Sus Score?? also allows you to verify the expected value—simply multiply each level by its probability and sum the results. This is useful for checking that the level assigned by the system aligns with the shape of the underlying distribution.
## Batching for Speed and Cost Efficiency
Because each question is evaluated independently within a request, you can bundle many questions into a single call. The state travels across the network once, and the model evaluates all questions against it in parallel. Compared to making individual calls for each question, this dramatically reduces both latency and token consumption. The savings come from sending the context only once rather than repeatedly, and from the overhead of connection setup being amortized across all questions in the batch.
This pattern is particularly powerful for speculative evaluation—asking every question you might need to know, even if some answers turn out to be irrelevant to the final decision. Since questions cannot see each other, there is no risk of cross-contamination.
## Routing with Confidence Thresholds
A judgment is only as useful as the action it triggers. One practical pattern involves confidence-gated routing, where different decision thresholds apply depending on the stakes involved. For low-risk actions like viewing a customer’s balance, a modest confidence threshold may suffice. For high-risk actions like closing an account or approving a large transfer, the bar rises significantly.
The thresholds live as ordinary code values, not as hidden instructions buried inside a prompt. This means they can be reviewed, versioned, tested, and adjusted like any other piece of business logic. Anything that falls into an unrecognized category or fails to meet its confidence threshold can be routed to a human reviewer, creating a safety net that scales with the sensitivity of the action.
## Composite Scoring: Separating Evaluation from Priorities
Another powerful pattern is composite scoring, where the model makes multiple independent judgments about a single subject, and the code applies weighting to produce a final ranking. For example, when evaluating candidates, each person’s biography can be assessed across several dimensions—technical depth, systems experience, leadership evidence, and communication skills. Each dimension receives a normalized score, and then separate weight vectors apply different priorities for different roles.
Because the model’s judgments are stored separately from the policy weights, changing what you value instantly reshuffles the ranking without requiring any additional inference. This separation of evaluation from policy is a significant architectural advantage.
## Typed Function Calling Through Closed Sets
Function calling becomes more reliable when it is decomposed into closed-set questions. Instead of asking a model to output a function name and arguments in natural language and then parsing the response, you can define every possible function and every possible argument as a selection question. One question picks the tool, another picks the target, and additional questions pick the specific parameters relevant to that tool. Only the questions corresponding to the selected tool are read by the calling code, and the weakest of all the relevant confidence scores can serve as the overall confidence of the function call.
This approach ensures that the arguments passed to any function are drawn from a known, safe set—no unexpected strings, no hallucinated field values.
## Counting Limitations and Workarounds
A documented limitation of these models is that they are not consistently reliable at counting within a single question. Rather than trusting a model to tell you “how many items in this list meet a criterion,” the recommended pattern is to ask one binary Noul question per item and sum the resulting probabilities in your own code. This trades a single unreliable question for several reliable ones, and the summation happens in code where it is precise and verifiable.
## Production Considerations
Moving from experimental notebooks to production services requires several architectural layers. Typed response models—defined by subclassing a base response type and declaring the expected answers as fields—give you attribute access with validation built in, rather than dealing with raw dictionaries. An asynchronous client can dispatch many requests concurrently, dramatically improving throughput for batch workloads. Retry policies with configurable backoff and timeout bounds protect against transient failures without creating unbounded latency.
Errors themselves are typed—the system rejects malformed requests before they reach the model, and unknown model identifiers are surfaced as distinct error types carrying HTTP status information so your error-handling code can respond appropriately.
A running ledger that tracks every call, its token usage, and its cost ensures that you can price the entire pipeline. Since the input token price is published and output tokens are typically free in this architecture, cost estimation becomes straightforward arithmetic.
—
## Frequently Asked Questions
**Q: How is this different from using a standard LLM with structured output prompts?**
A: Traditional LLMs generate free-form text, and even with careful prompting, the interpretation of that text introduces variability and parsing fragility. This system returns bounded, typed values directly—probabilities over a known set of options—so there is no intermediate step where text must be interpreted or regex-parsed. The output shape is guaranteed by the question definitions, not by the model’s prose style.
**Q: Can I ask questions that are not pre-defined?**
A: The system is designed around pre-defined questions with known answer spaces. This is a deliberate constraint that ensures reliability. If you need a genuinely open-ended response, a text-generating model would be more appropriate. The strength of this approach lies in its precision for bounded decision tasks.
**Q: Does the model see other questions when they are batched together?**
A: No. Each question is evaluated in isolation from every other question, even within the same request. You can verify this by comparing answers to the same question when asked alone versus when asked alongside other questions—the results should be identical.
**Q: How do I know what confidence threshold to use for routing?**
A: Confidence thresholds should be determined by evaluating the model’s performance on your specific data and aligning them with the cost of being wrong. Low-stakes decisions can tolerate lower thresholds; high-stakes decisions require higher ones. The key advantage is that these thresholds live in code and can be continuously monitored and adjusted.
**Q: Is this system a replacement for prompt engineering?**
A: It is a different paradigm rather than a replacement. Prompt engineering optimizes the text generation path. This system replaces that path entirely with structured question answering. The skill shifts from crafting prompts to designing good questions, criteria, and policies—which are all defined in code and can be tested systematically.
**Q: What happens if the model returns low confidence on all options?**
A: Low confidence across all options is itself a useful signal. It indicates the model is uncertain about the input state or that the criteria do not cleanly apply. In a routing system, such cases can be automatically escalated to human review, ensuring that ambiguous inputs do not trigger irreversible actions.
—
## Conclusion
The shift from text generation to structured judgment representation represents a meaningful advance for AI systems that serve as components in larger decision pipelines. By returning typed probabilities over closed sets of options, the model becomes a source of small, composable decisions that code can branch on, weight, aggregate, and route—without the fragile text-to-structure translation step that so often breaks in production.
Every answer arrives as either a label, a level on a rubric, or a yes/no probability with its distribution attached. This makes it possible to set concrete thresholds, define explicit policies, and trace every decision back to the dimension that produced it. Batching reduces cost and latency; confidence-gated routing adds safety; composite scoring decouples evaluation from priorities; and typed function calling turns natural language into validated function invocations.
The production-grade engineering pieces—typed response models, asynchronous fan-out, retry policies, error handling, and cost tracking—are all small and composable. Together, they form a complete decision service architecture. What remains is the human work no SDK can automate: designing the right questions, defining fair and accurate criteria, setting appropriate thresholds, and continuously evaluating performance against real-world outcomes.
Thank you for reading
