How I Built a Smart Ticket Search System Using PyTorch and GloVe

2026-03-17 · Source: NLP on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

A developer built a smart ticket search system using PyTorch and GloVe to find similar customer support tickets from a dataset of 8,469 complaints. The system processes text categories using Label Encoding for priority and One-Hot Encoding for channel, both implemented from scratch. It features a custom TF-IDF component with a regex tokenizer, a 5,000-word vocabulary, and bigram/trigram generators, storing scores as sparse tensors. Semantic understanding is achieved using 300-dimensional GloVe embeddings, with out-of-vocabulary words handled by random normal vectors and TF-IDF weighted averaging. A hybrid search formula combines 40% TF-IDF and 60% GloVe scores. Optimized on dual Kaggle T4 GPUs, the system processes 100 queries in 0.141 seconds, achieving an average of 1.41ms per query and a Precision@5 of 21.10%. An interactive Gradio web app allows users to query and adjust search parameters.

Key takeaway

For NLP Engineers building custom search or recommendation systems, implementing core components like TF-IDF and GloVe from scratch provides deeper algorithmic understanding and fine-grained control. You should consider a hybrid approach (e.g., 40% TF-IDF, 60% GloVe) to balance exact keyword matching with semantic understanding, especially when query speed and accuracy on large datasets are critical.

Key insights

Combining TF-IDF with GloVe embeddings creates a robust hybrid search for semantic and keyword matching.

Principles

• Rare words carry more TF-IDF weight.
• GloVe captures semantic meaning better than TF-IDF.
• Weighted averaging improves sentence vector quality.

Method

Implement custom regex tokenization, build a top-5000 word vocabulary, compute TF-IDF scores, load 300-dim GloVe vectors, and combine TF-IDF and GloVe scores with a 0.4:0.6 weighting for hybrid search.

In practice

• Use `torch.nn.DataParallel` for GPU scaling.
• Handle OOV words with random normal vectors.
• Build interactive apps with Gradio.

Topics

• Natural Language Processing
• Semantic Search
• TF-IDF
• GloVe Embeddings
• PyTorch

Code references

• hashimrana478-bot/-HSRIS-NLP-Assignment

Best for: AI Student, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Building a Hybrid Semantic Search Engine from Scratch: A Deep Dive into TF-IDF, GloVe, and Dual GPU… — Hybrid search combining TF-IDF and GloVe embeddings significantly improves intent-based query relevance over keyword-only methods.
• When I started this assignment I honestly had no idea where to begin. — Combining TF-IDF and GloVe embeddings improves semantic search accuracy for customer support tickets.
• The Engine Behind Information Retrieval: Exploring TF-IDF — TF-IDF quantifies word importance by balancing local frequency with global rarity, enabling effective information retrieval.
• From TF-IDF to Transformers: Implementing Four Generations of Semantic Search — Semantic search evolved from explicit rules to learned contextual representations, balancing interpretability with semantic depth.
• Building Context-Aware Search in Python with LLM Embeddings + Metadata — Context-aware semantic search combines embedding-based similarity with metadata filtering for relevant results respecting contextual constraints.
• SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain — Specialized benchmarks are crucial for evaluating MLLMs in knowledge-intensive, visually ambiguous short-video domains.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NLP on Medium.