Support Ticket Triage with NLP: TF-IDF and a Linear Model, Scored on Tickets You Can't See

Kesar Cart, an online home-and-kitchen store in Indore, gets a few hundred support tickets a day. One person reads every ticket and forwards it to one of five desks: payments, delivery, returns, account or technical. Tickets wait hours, and some bounce between desks. Your job is to build the router that reads a ticket and sends it to the right desk.You get your own copy of Kesar Cart's tickets: two months with the desk that handled each one, and a week of new tickets without it. You build the router yourself, lesson by lesson. Then you give the same job to your AI assistant (ChatGPT, Claude, Gemini or Copilot) and check what it gives back. Bartman runs your script in a sealed sandbox and grades your scores, desk by desk, against where those new tickets actually belonged. Neither you, your AI nor the sandbox ever sees those desks.<ul><li>Clean a real helpdesk export: repeated rows, a hand-typed label, and why a majority-desk baseline is the floor, not a start.</li><li>Implement a production cleaning contract with regular expressions: lowercase first, then mask order ids, phone numbers, emails, amounts and links.</li><li>Understand bag of words, document frequency and scikit-learn's IDF well enough to compute them by hand and check them against TfidfVectorizer.</li><li>Catch leakage: a resolution note written after the ticket was routed, which makes any model look perfect.</li><li>Train TF-IDF + logistic regression and complement naive Bayes in pipelines, compare them with cross-validated macro-F1, auto-route at 50% confidence and fill a human review queue.</li></ul>Every learner gets different tickets, so answers can't be copied. Each check explains what went wrong without giving the answer away, and the classic mistakes (cleaning steps in the wrong order, a vocabulary fitted on next week's tickets, a leaky column, a constant guess) are recognised and named.

Tools: Python, pandas, scikit-learn, NLP, Regular Expressions, TF-IDF, Logistic Regression, Naive Bayes, Cross-Validation, Data Leakage, matplotlib, AI Assistants for Python