Panels February 23–27, 2013, San Antonio, Texas, USA Community, Impact and Credit: Where should I submit my papers? Abstract Aaron Halfaker R. Stuart Geiger We (the authors of CSCWs program) have finite time and GroupLens Research School of Information energy that can be invested into our publications and the University of Minnesota University of CA, Berkeley research communities we value. While we want our work [email protected] [email protected] to have the most impact possible, we also want to grow Cliff Lampe Loren Terveen and support productive research communities within School of Information GroupLens Research which to have this impact. This panel discussion explores Michigan State University University of Minnesota the costs and benefits of submitting papers to various [email protected] [email protected] tiers of conferences and journals surrounding CSCW and Amy Bruckman Brian Keegan reflects on the value of investing hours into building up a College of Computing Northeastern University research community. Georgia Inst. of Technology [email protected] [email protected] Author Keywords Aniket Kittur Geraldine Fitzpatrick community; credit; impact; publishing; peer review HCI Institute Vienna University of Carnegie Mellon University Technology ACM Classification Keywords [email protected] geraldine.fi[email protected] H.5.0. [Information Interfaces and Presentation (e.g. HCI)]: General Introduction We (the authors of CSCWs program) have finite time and energy that can be invested into our publications and the research communities we value. In order to allow our work to have an impact, we must also grow and maintain Copyright is held by the author/owner(s). productive research communities within which to share CSCW ’13 Companion, Feb. 23–27, 2013, San Antonio, Texas, USA. our work. This panel discussion explores the costs and ACM 978-1-4503-1332-2/13/02. benefits of submitting papers to various tiers of 89 Panels February 23–27, 2013, San Antonio, Texas, USA conferences and journals surrounding CSCW and reflects theoretically, decreases the activation energy necessary for Format: on the value of investing hours into building up a research forming new collaborations. But the way that conferences community. By comparing CSCW to restrictive, top tier accept and reject papers can have an effect on the type of • The panel presentation but general conferences like CHI and less restrictive, communities that are formed. will start with a short specialized conferences like WikiSym and RecSys, the introduction by the panelists will frame a discussion around the value of The first-round rejection process for most conference moderators. participation in different types of research communities proceedings encourages rejected submissions to be and how decisions on where to invest ones time affect resubmitted to a different conference { if not, we must • Each panelist will follow metrics used to evaluate our worth. wait almost a year before resubmission. A common but with a presentation of often unacknowledged practice is to therefore to resubmit their perspective on an This discussion is valuable because it provides an rejected papers to slightly or completely different aspect of the panels opportunity for experienced researchers to advise young conferences. Substantive comments about the quality of themes. researchers on which communities to invest their time. It the paper obviously ought to be addressed in revisions, will also provide an opportunity to address several trends but how much do we really think about revising the same • The moderators will relevant to the CSCW community: paper for a different conference? guide a discussion between the audience If our conferences are, as is commonly claimed, journals and panelists. • The merging role of top conferences and journals that meet once a year, what implications does the debate • The value of small, specialized conferences over revise and resubmit versus first-round rejection have for the boundaries and bridges between research • Inconsistencies between impact and impact metrics communities? If we are more like a journal and encourage revisions to be resubmitted, there is an argument that this To focus the discussion, panelists will address three would make authors target their submissions towards a themes of publication: community, impact and credit. particular venue. Yet is this a good thing or a bad thing? Each of these themes will be discussed both in terms of Does first-round rejection encourage authors to form their current state and where each panelist thinks they communities which are not coextensive with conferences ought to be. and are instead more ad-hoc around other areas of commonality? Or does this effectively silo similar kinds of Themes research into areas that do not interact with others in the Community same conference (Wikipedia research at CSCW, for How does the structure of research publication example). affect the kinds of communities we form? On the one hand, we can think of communities as existing around a single conference that extends outward (e.g. the Conferences are designed to bring people to the same CSCW, CHI, RecSys, or ICTD community), but we can physical space. This allows members of a research also think of communities as existing around other community to meet face-to-face to discuss their work, and common areas, such as common methodologies, 90 Panels February 23–27, 2013, San Antonio, Texas, USA theoretical approaches, and specific applications? increase the required investment required by reviewers but also allow for reviewers to have a more substantial impact on the quality of a program. • How does the structure of publication affect the kinds of communities that we form? • Where should researchers direct their effort? • How are the boundaries between research communities be drawn? { . for the benefit of their work? • Does the community have a conference or does a { . for the good of science? conference form a community? Credit Impact How do we measure the worth of a researcher What is impact and what types of impact can or a publication venue? researchers have? For better or worse, hiring and tenure committees often When we talk about impact, we usually mean moving the use impact metrics to judge the worthiness of applicants. field forward via publications of important research. The lack of an impressive citation count can dramatically However, the impact of a publication is not solely affect a researchers career. In this way, metrics dont just dependant on the quality of the research and manuscript, describe the pattern of publishing, but direct it as well. but also the audience to whom it is presented. In the ecosphere of conferences and journals, a line might be The quality of a conferences are also commonly evaluated drawn between small conferences with a limited focus and by a simple metric: acceptance rate. While acceptance large conferences with a broader focus. rate tends be an effective measure of the restrictiveness of conferences and journals that use similar review patterns, Submissions to smaller conferences may have fewer the metric ceases to be useful when comparing a potential readers, they also offer direct address to a tight conference with a revise and resubmit pattern to one with community which is more likely to respond productively to first round rejections only. Assuming that a revise and the submission. Conversely, papers published to large resubmit process is beneficial to the research community, conferences and journals will usually see a larger audience how does one weigh such benefits against the cost of a of potential readers but a smaller proportion of readers are higher acceptance rate? likely to find the research useful. But these measurements of impact do not capture all Further, not all impact happens via paper publications. meaningful contributions. As was mentioned above, not Service work like reviewing papers and being part of the all ways that a researcher can have impact in a field result program committee is necessary for a functioning peer in a paper with a lot of citations. When the measurement review system. Processes like revise and resubmit both does not capture the value of a contribution, it is likely to 91 Panels February 23–27, 2013, San Antonio, Texas, USA misdirect our efforts towards less worthy end. In this case, final decision to accept some and reject others feels are we obligated to ignore the metric. arbitrary; authors sometimes find their favorite papers do not make it in to the top conferences, but when published What are some contributions that we make, but do not at less prestigious conferences receive an enthusiastic get credit for? Is it important that reviewers receive credit response. Attending less selective conferences may expose for reviewing papers? How might we change our one to more relevant and interesting work and offer much measurements of impact to direct researchers better? greater networking and community building opportunities, and high rejection rates can lead individuals and whole Panelists sub-communities to stop participating in a conference. Loren Terveen In short, lots of people think the current publication Bio Loren Terveen is a professor in the Department of culture is broken. Within the past few years, a number of Computer Science and Enginereing at the University of conferences have experimented with alternative models, Minnesota. He specializes in human-computer interaction like CSCW's 2-phase-with-a-full-revision-cycle approach. and computer-mediated communication. His academic However, these alternatives raise challenges of their own, record includes about 50 refereed journal and conference both logistical (do they scale?) and evaluative (if the papers, one book, four book chapters, nine U.S. Patents, acceptance rate increases, how do we convince people in addition to numerous other publications. that our conference is still high quality?). I will talk briefly about some of the alternative models being considered Position Over the past 20 years, Computer Science and their pros and cons.