Pay-Per-Question: Towards Targeted Q&A with Payments
Total Page:16
File Type:pdf, Size:1020Kb
Pay-per-Question: Towards Targeted Q&A with Payments Steve T.K. Jan, Chun Wang Qing Zhang Gang Wang Computer Science School of Education Computer Science Virginia Tech Virginia Tech Virginia Tech Blacksburg, VA, USA, 24060 Blacksburg, VA, USA, 24060 Blacksburg, VA, USA, 24060 {tekang, wchun}@vt.edu [email protected] [email protected] ABSTRACT Fenda is a social network app that connects users to well- Online question and answer (Q&A) services are facing key known domain experts and celebrities to ask questions with challenges to motivate domain experts to provide quick and payments. Launched in May 2016, Fenda quickly gained 10 high-quality answers. Recent systems seek to engage real- million registered users, 500K paid questions, and 2 million world experts by allowing them to set a price on their answers. US dollar revenue in the first 2 months [44]. The success This leads to a “targeted” Q&A model where users to ask of Fenda has created a new wave of payment-based Q&A questions to a target expert by paying the price. In this pa- services in China (Zhihu, DeDao, Weibo QA) and the U.S. per, we perform a case study on two emerging targeted Q&A (Whale, Campfire.fm, Yam). systems Fenda (China) and Whale (US) to understand how Fenda focuses on verified, real-world domain experts, which monetary incentives affect user behavior. By analyzing a large is different from earlier payment-based Q&A services driven dataset of 220K questions (worth 1 million USD), we find that by an anonymous crowd (e.g., Google Answers, ChaCha [2, payments indeed enable quick answers from experts, but also 11, 19]). More specifically, Fenda uses a targeted model where drive certain users to game the system for profits. In addition, users ask questions to a target expert by paying the question this model requires users (experts) to proactively adjust their fee set by the expert. This model seeks to better engage and price to make profits. People who are unwilling to lower their motivate experts. In addition, Fenda is the first system that prices are likely to hurt their income and engagement over explicitly rewards people for asking good questions. After a time. question is answered, other users in the network need to pay a ACM Classification Keywords small amount ($0.14) to access to the answer. This “listening fee” will be split evenly between the question asker and the H.5.0 Information Interfaces and Presentation (e.g. HCI): Mis- answerer (Figure1). A good question may attract enough cellaneous; J.4 Computer Applications: Social and Behavioral listeners to compensate the initial question fee. Sciences In this paper, we seek to understand the effectiveness of the Author Keywords targeted Q&A model and the impact of monetary incentives Online Q&A Service; Crowdsourcing; Payments to the Q&A system. By performing a case study on Fenda and a U.S.-based system Whale [42], we explore the answers INTRODUCTION to a list of key questions: How does the question price af- The success of online question and answer (Q&A) services de- fect the answering speed? What is the potential problematic pends on the active participation of users, particularly domain user behavior caused by the monetary incentives? Whether experts. With highly engaging experts, services like Quora and how could the pricing behavior predict user income and and StackOverflow attract hundreds of millions of visitors engagement level? These questions are critical for payment- worldwide [47]. However, for most Q&A systems, domain based Q&A design, and Fenda and Whale provide a unique experts are answering questions voluntarily for free. As the opportunity to study them. question volume going up, it becomes difficult to draw experts’ attention to a particular question, let alone getting answers For our analysis, we collected a large dataset of 88,540 users on-demand [34]. and 212,082 answers from Fenda (2 months in 2016), and 1,419 users and 9,199 answers from Whale (6 months in 2016– To motivate domain experts, one possible direction is to in- 2017), involving over 1 million dollar transactions. Our analy- troduce monetary incentives [10]. Recently, a payment-based sis makes three key findings: Q&A service called Fenda [4] is rising quickly in China. • First, using the new incentive model, both Fenda and Permission to make digital or hard copies of all or part of this work for personal or Whale successfully attract a small group of high-profile classroom use is granted without fee provided that copies are not made or distributed experts who make significant contributions to the commu- for profit or commercial advantage and that copies bear this notice and the full citation nity. Fenda experts count for 0.5% of the user population, on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, but have contributed a quarter of all answers and nearly half to post on servers or to redistribute to lists, requires prior specific permission and/or a of the revenue. fee. Request permissions from [email protected]. GROUP 2018, January 7–10, 2018, Sanibel Island, FL, USA • Second, the incentive model has a mixed impact on user Copyright © 2018 Association of Computing Machinery. ACM ISBN 978-1-4503-5562-9/18/01 ...$15.00. behavior. Monetary incentive enables quick answers (av- http://dx.doi.org/10.1145/3148330.3148332 erage delay 10–23 hours) and motivates users to ask good Service Q&A Model Fee? Mobile? Content Fenda Targeted Y Y Text/Audio questions. However, we find a small number of manipula- Whale Targeted Y Y Text/Video tive users who either aggressively ask questions to make Jisiklog Crowdsource Y Y Text money from listeners, or collude/collaborate to improve ChaCha Crowdsource Y Y Text their perceived popularity. Google Answer Crowdsource Y N Text Mahalo Answer Crowdsource Y N Text • Third, we find that different pricing strategies of users Naver Q&A Crowdsource N N Text (question answerers) can affect their own engagement level. Quora Crowdsource N N Text Yahoo Answer Crowdsource N N Text Users who proactively adjust their price are more likely to StackOverflow Crowdsource N N Text increase income and engagement level. However, certain Table 1. Fenda/Whale vs. other Q&A services. celebrities are unwilling to lower their price, which in turn hurts their income and social engagement. (most are defunct) are driven by an anonymous crowd instead To the best of our knowledge, this is the first empirical study of a social network that engages real-world experts. Users are on payment-based, targeted Q&A services. Our study pro- primarily driven by financial incentives without a strong sense vides practical guidelines for other arising payment-based of community [19, 10]. This is concerning since research Q&A services (Zhihu, DeDao, Campfire.fm, Yam) and reveals shows monetary incentive plays an important role in getting key implications for future online Q&A system design. We users started, but it is the social factors that contribute to the believe this is a first step towards understanding the economy persistent participation [28]. of community-based knowledge sharing. Researchers have studied the impact of monetary incentives RELATED WORK but the conclusions vary. Some researchers find that monetary Online Question Answering. In recent years, researchers incentives improve the answer quality [9] and the response have studied online Q&A services from various aspects [33]. rate [49]. Others suggest that payments merely reduce the Early studies have focused on identifying domain experts [26, response delay but have no significant impact on the answer 8] and routing user questions to the right experts [20, 27, 39]. quality [2, 12, 11]. Studies also show that payment-based Other works focused on assessing the quality of existing ques- Q&A can reduce low-quality questions since users are more tions and answers [36, 29, 45, 30, 37,1,9, 35] and detecting selective regarding what to ask [10, 11]. low quality (or even abusive) content [16]. Finally, researchers Mobile Q&A. Mobile Q&A services leverage the ubiquitous also studied Q&A activities in online social networks [25,7]. mobile devices to enable user-friendly Q&A experience [18, As the sizes of Q&A systems rapidly grow, it becomes chal- 19]. Systems like ChaCha and Jisiklog allow users to interact lenging to engage with experts for timely and high-quality with an online crowd via text messages. Fenda and Whale answers [34]. are also mobile-only Q&A services (their web interfaces are Crowdsourcing vs. Targeted Q&A. As shown in Table1, read-only). The questions in Fenda and Whale are still written most Q&A systems rely on crowdsourcing where any users in text, but the answers are recorded vide/audio messages. in the community can answer the question. Fenda and Whale adopt a targeted Q&A model where users can ask questions to RESEARCH QUESTIONS AND METHOD a target expert with payments. In this targeted Q&A model, it Systems like Fenda and Whale are leading the way to socially is the answerer (e.g., the expert) who has the upper hand to set engage with real-world experts for question answering. The the price for their answers. This differs Fenda and Whale from introduction of monetary incentives makes user interactions the earlier crowdsourcing Q&A services (e.g., Google An- even more complex. If not carefully designed, monetary incen- swers [2], and Mahalo [11]), and the broader crowdsourcing tives can lead the systems down to the wrong path with users marketplace (e.g., Mechanical Turk) [17].