Algorithmic Bias in Google Searches for Political Parties and Candidates
Johannes Nakayama, Nils Plettenberg, Laura Burbach, André Calero Valdez
Human-Computer Interaction Center, RWTH Aachen University The Datenspende BTW17 Dataset search terms
search terms …
database
search terms
4000 participants Ø 1647 searches per day parties: candidates:
‘CDU’ ‘Angela Merkel’ ‘CSU’ ‘Martin Schulz’ ‘SPD’ ‘Christian Lindner’ ‘FDP’ ‘Katrin Göring-Eckardt’ ‘Bündnis 90/Die Grünen’ ‘Cem Özdemir’ ‘Die Linke‘ ‘Sahra Wagenknecht’ ‘AfD’ ‘Dietmar Bartsch’ ‘Alice Weidel’ ‘Alexander Gauland’ Can we find evidence for personalization in the dataset? First approach
set intersection: first three/six/nine results A C
B D
C A
D G
E F
F B
G H
H E
I I A C
B D
C A
D G
E F
F B
G H
H E
I I A C
B D
C A
overlap of 66.67 %
repeat for every list with every other list
compute mean downside: only rough order effects
Rank-biased overlap (RBO) (Webber et al. 2010)
Result 1 Result 1 Result 2 Result 2 Result 3 Result 3 Result 4 4 Result 5 Result Result 5 Result 6 Result 6 Result 7 Result 7 Result 8 Result 8 How does RBO work?
Identical Lists Entirely different Same items, reverse items order User 1 User 2 User 1 User 2 User 1 User 2 A A A K A J B B B L B I C C C M C H D D D N D G E E E O E F F F F P F E G G G Q G D H H H R H C I I I S I B J J J T J A
RBO score: 1.0 RBO score: 0 RBO score: 0.5116076 Last 5 items different How does RBO work? User 1 User 2 A A B B 1st item different 3rd item different C C User 1 User 2 User 1 User 2 D D A K A A E E B B B B F F C C C K G G D D D D H H E E E E I I F F F F J J G G G G K P H H H H L Q I I I I M R J J J J N S RBO score: 0.7297158 RBO score: 0.8747158 O T RBO score: 0.901564 Similarity of Search Results
T-test
significant differences between RBO scores of parties and candidates
t = 27.4, p < .001
How can we explain the downward trend ?
Search Volume no absolute values only relative scores
Trends
Summary
significant difference between parties and candidates dependence between RBO scores and search volume Future Work
R T
RBO package location content analysis (already in progress) open data and all analyses available at: osf.io/e598k