Algorithmic Bias in Google Searches for Political Parties and Candidates

Johannes Nakayama, Nils Plettenberg, Laura Burbach, André Calero Valdez

Human-Computer Interaction Center, RWTH Aachen University The Datenspende BTW17 Dataset search terms

search terms …

database

search terms

4000 participants Ø 1647 searches per day parties: candidates:

‘CDU’ ‘’ ‘CSU’ ‘’ ‘SPD’ ‘’ ‘FDP’ ‘Katrin Göring-Eckardt’ ‘Bündnis 90/Die Grünen’ ‘Cem Özdemir’ ‘Die Linke‘ ‘’ ‘AfD’ ‘’ ‘’ ‘’ Can we find evidence for personalization in the dataset? First approach

set intersection: first three/six/nine results A C

B D

C A

D G

E F

F B

G H

H E

I I A C

B D

C A

D G

E F

F B

G H

H E

I I A C

B D

C A

overlap of 66.67 %

repeat for every list with every other list

compute mean downside: only rough order effects

Rank-biased overlap (RBO) (Webber et al. 2010)

Result 1 Result 1 Result 2 Result 2 Result 3 Result 3 Result 4 4 Result 5 Result Result 5 Result 6 Result 6 Result 7 Result 7 Result 8 Result 8 How does RBO work?

Identical Lists Entirely different Same items, reverse items order User 1 User 2 User 1 User 2 User 1 User 2 A A A K A J B B B L B I C C C M C H D D D N D G E E E O E F F F F P F E G G G Q G D H H H R H C I I I S I B J J J T J A

RBO score: 1.0 RBO score: 0 RBO score: 0.5116076 Last 5 items different How does RBO work? User 1 User 2 A A B B 1st item different 3rd item different C C User 1 User 2 User 1 User 2 D D A K A A E E B B B B F F C C C K G G D D D D H H E E E E I I F F F F J J G G G G K P H H H H L Q I I I I M R J J J J N S RBO score: 0.7297158 RBO score: 0.8747158 O T RBO score: 0.901564 Similarity of Search Results

T-test

significant differences between RBO scores of parties and candidates

t = 27.4, p < .001

How can we explain the downward trend ?

Search Volume no absolute values only relative scores

Trends

Summary

significant difference between parties and candidates dependence between RBO scores and search volume Future Work

R T

RBO package location content analysis (already in progress) open data and all analyses available at: osf.io/e598k