## Learning a Meaningful Severity Score from Limited Outcomes with Neural Networks
In many real-world machine learning tasks, especially in healthcare, the available training data is not as detailed as we would like. Consider a scenario where you have data on patients who acquired “Pathogen A” and suffered from an infectious respiratory illness. For each patient, you have eight features—such as age, smoking history, comorbidities, and vital deviations—and one of three outcomes: (a) treated at home and recovered, (b) hospitalized and recovered, or (c) died.
It is relatively straightforward to train a neural network to predict which of these three outcomes a patient will experience with very high accuracy. However, health authorities often need more granular insight: among patients who are eligible for home treatment, who is most at risk of needing hospitalization? Among those predicted to be hospitalized, who is most at risk of death? In other words, can we assign a continuous severity score that reflects how serious an infection is likely to be?
This article walks through the design of a neural network architecture that learns such a score using only categorical outcome labels. The approach combines a low‑capacity regression backbone with a lightweight “head” that interprets the score in terms of clinically meaningful categories. The result is a model that is both interpretable and practical for deployment.
—
### The Dataset
To demonstrate the method, a synthetic dataset was generated based on eight input features:
– Previous infection with Pathogen A (boolean)
– Previous infection with Pathogen B (boolean)
– Acute/current infection with Pathogen B (boolean)
– Cancer diagnosis (boolean)
– Weight deviation from average (−100 to 100)
– Age in years (0–100)
– Blood pressure deviation from average (0–100)
– Years smoked (0–~88)
These features were sampled uniformly, with special handling to ensure that approximately 50% of patients were non‑smokers and that smoking duration correlated realistically with age. The outcomes were designed to occur with roughly equal probability.
When plotted by age and weight, the three outcome classes show clear separation, illustrating the non‑linear but deterministic nature of the toy problem.
—
### The Limitations of Direct Score Prediction
A natural first attempt is to map the three outcomes to numeric values—0 for home treatment, 1 for hospitalization, and 2 for death—and train a single‑output neural network using mean squared error. While this works in the sense that the model associates higher numbers with worse outcomes, the resulting predictions tend to collapse toward the class centers. Instead of a smooth score, the model outputs values clustered near 0, 1, and 2.
This behavior reflects the model’s high capacity to rearrange the input space to fit discrete targets, which is undesirable when we want a graded measure of severity.
—
### Step 1: Using a Low‑Capacity Regressor
To obtain a continuous score, the model’s capacity must be restricted so it cannot “cheat” by clustering outputs. The simplest effective approach is a linear regression—a single dense layer with no activation function.
This has several advantages:
– The output is a weighted linear combination of the inputs.
– The model is highly interpretable.
– The score can vary continuously rather than jumping between three fixed values.
However, a pure linear regressor cannot by itself enforce category boundaries that match the desired clinical ordering. We therefore introduce an additional component to the architecture.
—
### Step 2: Adding a Category Approximator Head
The key idea is to keep the linear regression as the encoder and add a lightweight decoder that predicts the categorical outcomes from the score. This decoder consists of a single dense layer with three outputs and a softmax activation, which learns to approximate the probabilities of each category given the score.
Training is performed using categorical cross‑entropy loss, so the model learns to associate ranges of scores with the correct outcomes. The encoder ensures that the score remains continuous, while the head ensures that those scores align with the observed categories.
—
### Step 3: Extracting Thresholds from the Model
Because the decoder uses a single linear transformation followed by softmax, its three output lines can be written explicitly as:
– o1 = w1·score + b1
– o2 = w2·score + b2
– o3 = w3·score + b3
The predicted category is the one with the highest output. The boundaries between categories occur where these lines intersect, which leads to explicit formulas for the thresholds:
– t0 = (b2 − b1) / (w1 − w2)
– t1 = (b3 − b2) / (w2 − w3)
These thresholds tell us, for example, the score above which a patient should be considered at risk of hospitalization or death.
—
### Step 4: Enforcing a Clinically Sensible Order
One subtlety is that the model is free to assign weights in any order, which could lead to nonsensical score interpretations (e.g., higher weights predicting lower risk). To prevent this, a “propagate‑sum” strategy is applied.
Instead of using the raw outputs of the dense layer, the three components are combined so that:
– o1 = d1
– o2 = d1 + d2
– o3 = d1 + d2 + d3
This guarantees that the effective weights influencing each category are monotonically increasing, aligning the ordering of outcomes with the natural progression from recovery to death.
—
### Step 5: Training and Evaluation
With the complete architecture in place, the model can be trained using standard optimization methods. After training, the learned weights and thresholds can be printed, allowing direct clinical interpretation.
In experiments, the linear bottleneck model achieved around 80% accuracy on the toy dataset—good given the simplicity of the model and the non‑linearity of the data. More importantly, the output scores formed a smooth distribution, with patients near decision boundaries clearly flagged as uncertain.
The model also remains fast to train and inexpensive to deploy, since it involves only a handful of parameters.
—
### Frequently Asked Questions
**Q: Why not just use a standard classifier with softmax output?**
A: Standard classifiers treat classes as discrete categories and do not provide a continuous severity score. They also do not enforce ordering constraints, which can lead to illogical predictions.
**Q: What happens if the relationship between features and outcome is highly non‑linear?**
A: A purely linear model may only approximate the score well. In such cases, a more expressive encoder (e.g., a few non‑linear hidden layers) can be used, provided it does not become so powerful that it collapses the score into discrete clusters.
**Q: Is the “propagate‑sum” trick necessary?**
A: It is a simple and effective way to enforce monotonic weight ordering using standard Keras components, avoiding the need for custom constraints while preventing obviously incorrect score orderings.
**Q: Can this approach be extended to more than three outcome categories?**
A: Yes. The same principles apply, with additional thresholds and output lines needed to separate the categories.
—
### Conclusion
Learning a meaningful severity score from categorical outcomes is challenging but feasible with carefully designed architectures. By combining a low‑capacity linear regressor with a category‑aware decoder, we obtain a continuous, interpretable score that reflects clinical intuition.
The resulting models are fast to train, easy to interpret, and suitable for environments where transparency and simplicity are as important as accuracy. In many high‑stakes domains, such modest models offer the right balance between predictive power and practical utility.
