Title:
An NLP-Driven Intelligent Video Query System for Interactive Video Retrieval
Authors:
Shailendra Singh Kathait (Co-Founder & Chief Data Scientist, Valiance Solutions)Ashish Kumar (Principal Data Scientist, Valiance Solutions)Samay Sawal (Intern Data Scientist, Valiance Solutions)
Summary:
Urban traffic surveillance produces vast quantities of continuous video data, but extracting actionable insights in real time remains a persistent challenge. This paper presents a novel Video Query System (VQS) that unifies deep-learning–based violation detection with natural language understanding to enable intuitive, efficient retrieval of relevant video segments from continuous CCTV feeds. Unlike traditional pipelines that operate in isolated silos, VQS consolidates multiple violation detectors — covering helmet non-compliance, illegal parking, over-speeding, wrong-way driving, and pedestrian tracking — into a single multi-attribute indexed repository. A large language model (LLM) translates free-form user queries (e.g., “show red motorcycles speeding above 60 km/h between 6–8 AM”) into structured, filterable query specifications, enabling precise retrieval and ranking of matching video frames with annotated metadata. The system’s interactive dashboard presents results as ranked frame sequences with bounding boxes and metadata, significantly accelerating investigative workflows for traffic management and enforcement authorities. Evaluation on real-world urban footage demonstrates strong detection precision (over 90%) and robust query interpretation, validating that natural-language–driven video querying bridges the gap between complex analytics and non-technical operators. The paper also discusses system design, user interaction, scalability, limitations, and future extensions for broader smart-city applications.
