Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Classification and Routing Prompts: Label Sets, Label Definitions, Choosing Few-Shot Examples, Ambiguous Cases and Confidence
A classification prompt's accuracy depends most on mutually exclusive label definitions, an enum in the schema and a "send to a person" label; the model's own confidence does not separate wrong answers. On Gül's 24-ticket set, definitions raised the score from 14/24 to 22/24, few-shot added nothing, qwen3:4b changed three tickets when the label order was reversed, and agreement between two small models sent both wrong answers to a person.
Classification and Its Metrics: Confusion Matrix, Precision, Recall, F1 and ROC-AUC
On imbalanced data accuracy misleads; a classification model has to be measured with precision, recall, PR-AUC and a threshold chosen from business costs. We predict Bookstore's order cancellation risk with logistic regression and FastTree and show with real output how 95.8% accuracy catches no cancellation at all, how to pick the threshold by cost on a validation set, how to check calibration, and a multiclass model that predicts the topic of a review.