ARTÍCULO
TITULO

Improving Natural Language Person Description Search from Videos with Language Model Fine-Tuning and Approximate Nearest Neighbor

Sumeth Yuenyong and Konlakorn Wongpatikaseree    

Resumen

Due to the ubiquitous nature of CCTV cameras that record continuously, there is a large amount of video data that are unstructured. Often, when these recordings have to be reviewed, it is to look for a specific person that fits a certain description. Currently, this is achieved by manual inspection of the videos, which is both time-consuming and labor-intensive. While person description search is not a new topic, in this work, we made two contributions. First, we improve upon the existing state-of-the-art by proposing unsupervised finetuning on the language model that forms a main part of the text branch of person description search models. This led to higher recall values on the standard dataset. The second contribution is that we engineered a complete pipeline from video files to fast searchable objects. Due to the use of an approximate nearest neighbor search and some model optimizations, a person description search can be performed such that the result is available immediately when deployed on a standard PC with no GPU, allowing an interactive search. We demonstrated the effectiveness of the system on new data and showed that most people in the videos can be successfully discovered by the search.

 Artículos similares

       
 
Zhixi Hu, Yi Zhu, Xiaoying Chen and Yu Zhao    
Autonomous driving is a safety-critical system, and the occupancy of its environmental resources affects the safety of autonomous driving. In view of the lack of safety verification of environmental resource occupation rules in autonomous driving, this p... ver más
Revista: Future Internet

 
Massimiliano Lo Turco, Elisabetta Caterina Giovannini and Andrea Tomalini    
In recent years we have been experiencing an ever-increasing number of Building Modeling Modeling (BIM) and Visual Programming Language (VPL) approaches in the architectural design field. These experiments have inspired new research strictly focused on e... ver más

 
Omar Doukari, Boubacar Seck, David Greenwood, Haibo Feng and Mohamad Kassem    
Buildings have a significant impact on energy consumption and carbon emissions. Smart buildings are deemed to play a crucial role in improving the energy performance of buildings and cities. Managing a smart building requires the modelling of data concer... ver más
Revista: Buildings

 
Alexander Hohl and Aynaz Lotfata    
The pandemic?s lockdown has made physical inactivity unavoidable, forcing many people to work from home and increasing the sedentary nature of their lifestyle. The link between spatial and socio-environmental dynamics and people?s levels of physical acti... ver más
Revista: Urban Science

 
Hugo Queiroz Abonizio, Janaina Ignacio de Morais, Gabriel Marques Tavares and Sylvio Barbon Junior    
Online Social Media (OSM) have been substantially transforming the process of spreading news, improving its speed, and reducing barriers toward reaching out to a broad audience. However, OSM are very limited in providing mechanisms to check the credibili... ver más
Revista: Future Internet