Geotag Propagation in Social Networks Based on User Trust Model

Ivan Ivanov, Peter Vajda, Jong-Seok Lee, Lutz Goldmann, Touradj Ebrahimi

Multimedia Signal Processing Group Ecole Polytechnique Federale de Lausanne, Switzerland

Multimedia Signal Processing Group Swiss Federal Institute of Technology Motivation 2

 We introduce users in our system for geotagging in order to simulate a real social network  GPS coordinates to derive geographical annotation, which are not available for the majority of web images and photos  A GPS sensor in a camera provides only the location of the photographer instead of that of the captured landmark  Sometimes GPS and Wi-Fi geotagging determine wrong location due to noise

http:// www.pl acecast .net

Multimedia Signal Processing Group Swiss Federal Institute of Technology Motivation 3

 Tag – short textual annotation (free-form keyword)usedto) used to describe photo in order to provide meaningful information about it  User-provided tags may sometimes be spam annotations given on purpose or wrong tags given by mistake  User can be “an algorithm”

http://code.google.com/p/spamcloud http://www.flickr.com/photos/scriptingnews/2229171225 Multimedia Signal Processing Group Swiss Federal Institute of Technology Goal 4

 Consider user trust information derived from users’ tagging behavior for the tag propagation  Build up an automatic tag propagation system in order to:  Decrease the annot ati on ti me, and  Increase the accuracy of the system

http://www.costadevault.com/blog/2010/03/listening-to-strangers

Multimedia Signal Processing Group Swiss Federal Institute of Technology Geotags 5

Tags • summer school • June • 2009 • sea Geotags • Hagia Sophia • Blue Mosque • Istanbul •Turkey • geo:lat = 41.01667  IPTC sc hema iiddis considered: • geo:lon = 28.96667  City – name of the city  Sublocation – area or name of the landmark  ElExamples: Ist anb ul (Hagi a S ophi a ), San Franc isco (Gold en Ga te Br idge )

Multimedia Signal Processing Group Swiss Federal Institute of Technology Trust and reputation systems 6

Email spam filtering

http://www.wpconfig.com/2010/07/04/tips-to-increase-your-pagerank Multimedia Signal Processing Group Swiss Federal Institute of Technology System overview 7

Multimedia Signal Processing Group Swiss Federal Institute of Technology Tag propagation 8

 Object duplicate detection module provides a matching score

matrix Si,Mj,Ni,j ,  1 ,  1   Two application scenarios:  Closed set probl em:

1if,SmaxS i,j  i,j  i,M1 Ci,j   0, otherwise

 Open set problem:

 ˆ 1if,SmaxSSS i,j  i,j i,j  i,M1 Oi,j   0, otherwise

Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling 9

 Three steps:  User evaluation – comparing the predicted tags to manually defined ground truth (for a real photo sharing system, such as Panoramio, it is not necessaryyg to collect ground truth data since user feedback can replace them): 1,j if user tags image i correctly Ui,j   0, otherwise  Trust model creation – the percentage of the correctly tagged images by particular user out of the overall number of tagged images M: M U T  i 1 iji ,j j M ˆ  Tag propagation – only tags from users who are trusted (TT j  ) are propagated to other photos in the dataset

Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling in Panoramio 10

Tagged incorrectly?

http://www.panoramio.com/photo/23286122

Multimedia Signal Processing Group Swiss Federal Institute of Technology User trust modeling in Panoramio 11

 Idea:

true flag (correct tag)

false flag (wrong tag + suggest a correct tag)

 The more mi spl acement s a user h as, th e more un trus ted he/she is  User trust ratio:

# true flags Tj   # all associated flags all images tagged by user j

Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 12

 Users can collaboratively eliminate a spammer:

Bob

Eve

Alice

T.  09 T.  07 T.  05 BbBob Alice Eve

Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 13

 Users can collaboratively eliminate a spammer:

Bob

Eve

Alice

T.  03 T.  07 T.  09 BbBob Alice Eve

Multimedia Signal Processing Group Swiss Federal Institute of Technology Eliminate spammers in Panoramio 14

 Users can collaboratively eliminate a spammer:

Bob

Eve

Alice

T.  07 T.  08 T.  04 BbBob Alice Eve

Multimedia Signal Processing Group Swiss Federal Institute of Technology Tag propagation based on user trust model 15