Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Model always predicts 1 or 0 on multiclass classification problem #15

#1

I have a multiclass classification problem for which Ludwig produces good models. However, it always predicts 1 or close to 1 in exactly one category and zero in everything else:

count 135257.000000
mean 0.999987
std 0.001505
min 0.754915
25% 1.000000
50% 1.000000
75% 1.000000
max 1.000000

In the above example, only two out 135,257 have a prediction value of less than 1.

I have trained dozens of these models, most with very successful results, but this is an issue in every case. I have tried changing the selection of input features and adjusting the confidence_penalty and robust_lambda parameters. Robust_lambda had a significant effect on the model performance, but still did not effect mean probability.

Has anyone else encountered this behavior, and does anyone have any other ideas how to fix it?

Thanks,
Craig

  • replies 1
  • views 2.1K
  • likes 1
cvarrichio · Author
#2

I have verified that I have the same issue while using a similar dataset on a related binary classification problem. 99% of the output probabilities are 1 or 0. I will work on producing a reproducible example.