Geotag Propagation in Social Networks Based on User Trust Model
Total Page:16
File Type:pdf, Size:1020Kb
1 Geotag Propagation in Social Networks Based on User Trust Model Ivan Ivanov, Peter Vajda, Jong-Seok Lee, Lutz Goldmann, Touradj Ebrahimi Multimedia Signal Processing Group Ecole Polytechnique Federale de Lausanne, Switzerland Multimedia Signal Processing Group Swiss Federal Institute of Technology Motivation 2 We introduce users in our system for geotagging in order to simulate a real social network GPS coordinates to derive geographical annotation, which are not available for the majority of web images and photos A GPS sensor in a camera provides only the location of the photographer instead of that of the captured landmark Sometimes GPS and Wi-Fi geotagging determine wrong location due to noise http: //www.pl acecas t.net Multimedia Signal Processing Group Swiss Federal Institute of Technology Motivation 3 Tag – short textual annotation (free-form keyword)usedto) used to describe photo in order to provide meaningful information about it User-provided tags may sometimes be spam annotations given on purpose or wrong tags given by mistake User can be “an algorithm” http://code.google.com/p/spamcloud http://www.flickr.com/photos/scriptingnews/2229171225 Multimedia Signal Processing Group Swiss Federal Institute of Technology Goal 4 Consider user trust information derived from users’ tagging behavior for the tag propagation Build up an automatic tag propagation system in order to: Decrease the anno ta tion time, and Increase the accuracy of the system http://www.costadevault.com/blog/2010/03/listening-to-strangers Multimedia Signal Processing Group Swiss Federal Institute of Technology Geotags 5 Tags • summer school • June • 2009 • sea Geotags • Hagia Sophia • Blue Mosque • Istanbul •Turkey • geo:lat = 41.01667 IPTC sc hema iiddis considered: • geo:lon = 28.96667 City – name of the city Sublocation – area or name of the landmark ElExamples: Is tanb u l (Hagi a S ophi a ), San Franc isco (Gold en Ga te Br idge ) Multimedia Signal Processing Group Swiss Federal Institute of Technology Trust and reputation systems 6 Email spam filtering http://www.wpconfig.com/2010/07/04/tips-to-increase-your-pagerank Multimedia Signal Processing Group Swiss Federal Institute of Technology System overview 7 Multimedia Signal Processing Group Swiss Federal Institute of Technology Tag propagation 8 Object duplicate detection module provides a matching score matrix Si,Mj,Ni,j , 1 , 1 Two application scenarios: Closed set probl em: 1if,SmaxS i,j i,j i,M1 Ci,j 0, otherwise Open set problem: ˆ 1if,SmaxSSS i,j i,j i,j i,M1 Oi,j 0, otherwise Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling 9 Three steps: User evaluation – comparing the predicted tags to manually defined ground truth (for a real photo sharing system, such as Panoramio, it is not necessaryyg to collect ground truth data since user feedback can replace them): 1,j if user tags image i correctly Ui,j 0, otherwise Trust model creation – the percentage of the correctly tagged images by particular user out of the overall number of tagged images M: M U T i 1 iji ,j j M ˆ Tag propagation – only tags from users who are trusted (TT j ) are propagated to other photos in the dataset Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling in Panoramio 10 Tagged incorrectly? http://www.panoramio.com/photo/23286122 Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling in Panoramio 11 Idea: true flag (correct tag) false flag (wrong tag + suggest a correct tag) The more mi spl acement s a user h as, th e more un trus te d he/she is User trust ratio: # true flags Tj # all associated flags all images tagged by user j Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 12 Users can collaboratively eliminate a spammer: Bob Eve Alice T. 09 T. 07 T. 05 BbBob Alice Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 13 Users can collaboratively eliminate a spammer: Bob Eve Alice T. 03 T. 07 T. 09 BbBob Alice Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 14 Users can collaboratively eliminate a spammer: Bob Eve Alice T. 07 T. 08 T. 04 BbBob Alice Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology Tag propagation based on user trust model 15 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 16 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Bob Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 17 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Bob Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 18 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Alice Bob Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 19 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Alice Bob Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 20 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Alice Bob Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Tag propagation based on user trust model 21 ˆ T 07. T 08. T 04. T 06. Bob Alice Eve Alice Bob Eve Multimedia Signal Processing Group Swiss Federal Institute of Technology http://www.flickr.com/photos/gustavog/9708628 Experiments 22 Dataset 1320 images ‒ 22 cities, 3 sublocations for each city, 20 sample images for each sublocation Multimedia Signal Processing Group Swiss Federal Institute of Technology Experiments 23 Dataset Large variety of views, distances and partial occlusions Multimedia Signal Processing Group Swiss Federal Institute of Technology Experiments 24 Methodology Extract a sub network from a large social network, in a way that every user in this subsystem annotates every monument in the subset of the dataset Upon this sub network, we build up an automatic propagation system in order to decrease the annotation time and increase the accuracy of the system 44 su bjec ts were as ke d to tag 66 p ho tos, c hosen in a dvance If either one of geotags (name of the area or name of the monument depicted in the image) is correlated with the object in the image, we assume that the imaggygge is correctly tagged Evaluation Recognition rate (accuracy): T RR A Multimedia Signal Processing Group Swiss Federal Institute of Technology Results 25 The recognition rate for all landmarks Multimedia Signal Processing Group Swiss Federal Institute of Technology Results 26 The recognition rate across the different locations groups (bars) and the recognition rate of all locations (dashed line) Multimedia Signal Processing Group Swiss Federal Institute of Technology Results 27 Trust ratios for the different users for a reduced (first 20 images) and the complete set of training images Multimedia Signal Processing Group Swiss Federal Institute of Technology Results 28 The recognition rate of the geotag propagation system and the percentage of the propagated tags versus the threshold involved in user trust model Multimedia Signal Processing Group Swiss Federal Institute of Technology Results 29 The recognition rate of the geotag propagation system versus the number of tags Multimedia Signal Processing Group Swiss Federal Institute of Technology Conclusion 30 An efficient system for automatic geotag propagation by associating locations with distinctive landmarks and using object duplicate detection for tag propagation User trust information derived from users’ tagging behavior for the tag propagation Only reliable geotags are propagated Increased accuracy of the tag propagation and a decrease of tagging efforts The proposed user trust model can be generalized to photo sharing platforms such as Panoramio Multimedia Signal Processing Group Swiss Federal Institute of Technology 31 Thank you for your attention! Ivan Ivanov, PhD student, EPFL [email protected] Multimedia Signal Processing Group Swiss Federal Institute of Technology.