TagSense: A Smartphone-based Approach to Automatic Image Tagging Chuan Qinyx∗ Xuan Baox Romit Roy Choudhuryx Srihari Nelakuditiy
[email protected] [email protected] [email protected] [email protected] yUniversity of South Carolina xDuke University Columbia, SC, USA Durham, NC, USA ABSTRACT 1. INTRODUCTION Mobile phones are becoming the convergent platform for per- Automatic image tagging has been a long standing problem. sonal sensing, computing, and communication. This paper While the fields of image processing and face recognition have attempts to exploit this convergence towards the problem of made significant progress, it remains difficult to automatically automatic image tagging. We envision TagSense, a mobile label a given picture. However, digital pictures and videos phone based collaborative system that senses the people, ac- are undergoing an explosion, especially with the proliferation tivity, and context in a picture, and merges them carefully to of high quality digital cameras embedded in mobile devices. create tags on-the-fly. The main challenge pertains to discrim- As these pictures get stored in online content warehouses, inating phone users that are in the picture from those that are the need to search and browse them is becoming crucial [1]. not. We deploy a prototype of TagSense on 8 Android phones, Furthermore, the growing sophistication in textual search is and demonstrate its effectiveness through 200 pictures, taken raising the expectations from image retrieval – users are ex- in various social settings. While research in face recognition pecting to search for pictures as they do for textual content. continues to improve image tagging, TagSense is an attempt Efforts to engage humans for labeling pictures (with crowd- to embrace additional dimensions of sensing towards this end sourcing or online gaming [2–5]) may be a stop-gap solution, goal.