Chemical Information Media in the Chemistry Lecture Hall: A

Home , Max Planck Society

Hot Topics CHIMIA 2010, 64, No. 5 309 doi:10.2533/chimia.2010.309 Chimia 64 (2010) 309–314 © Schweizerische Chemische Gesellschaft Chemical Information Media in the Chemistry Lecture Hall: A Comparative Assessment of Two Online Encyclopedias

Lukas Koroseca, Peter Andreas Limacherb, Hans Peter Lüthib, and Martin Paul Brändle*a

Abstract: The chemistry encyclopedia Römpp Online and the German universal encyclopedia Wikipedia were as- sessed by first-year university students on the basis of a set of 30 articles about chemical thermodynamics. Crite- ria with regard to both content and form were applied in the comparison; 619 ratings (48% participation rate) were returned. While both encyclopedias obtained very good marks and performed nearly equally with regard to their accuracy, the average overall mark for Wikipedia was better than for Römpp Online, which obtained lower marks with regard to completeness and length. Analysis of the results and participants’ comments shows that students attach importance to completeness, length and comprehensibility rather than accuracy, and also attribute less value to the availability of sources which validate an encyclopedia article. Both encyclopedias can be promoted as a starting reference to access a topic in chemistry. However, it is recommended that instructors should insist that students do not rely solely on encyclopedia texts, but use and cite primary literature in their reports. Keywords: Chemical information · Quality assessment · Römpp Online · Student survey · Wikipedia

Introduction however, raises the question whether the for American history,[1,7] or pharmacology.[8] chemical information in Wikipedia can be All came to the conclusion that the inves- In research as well as in studies, the use trusted, as freshman students without prior tigated reference works were more reliable of online media by graduate and under- knowledge about the system of chemical than Wikipedia. graduate students is common practice. literature, essential reference works, and Regarding chemistry content, an ex- In the chemistry library, we observe that databases in chemistry tend to do. How tensive comparison between the renowned students regularly use online encyclope- should chemistry lecturers and chemical chemistry encyclopedia Römpp Online dias as a quick and easy path to chemical information instructors respond? and German Wikipedia was done by Jana information, e.g. for substance informa- The reliability and accuracy of Wiki- Sonnenstuhl in her magister thesis[9] that tion with structures, molecular formulas, pedia articles is topic of an ongoing de- was accompanied by one of the authors safety information, physical properties, or bate,[1–9] because they are the collaborative (MB). Both quantitative comparisons re- for name reactions. Among the encyclope- result of many volunteers, amateurs as garding coverage, formal criteria (trans- dias, Wikipedia is most popular. The suc- well as experts. Wikipedia’s easy-to-use lations of headwords, number of sources, cess of Wikipedia can be attributed to the interface allows anyone to write and make further reading, graphics, and links) and fact that its articles are freely available on changes to articles. Authors can remain structural criteria (number of structure for- the Web and are highly interlinked, which anonymous, and there is no formalized mulae, tables, chapter titles, and reaction leads to a prominent ranking of its pages peer-review system in action. This open- equations) as well as an expert assessment in search engine results. For example, if ness, in the worst case, makes Wikipedia of the content of randomly selected articles one does a search on google.ch for the susceptible to vandalism.[1,2] The encyclo- were carried out. While Römpp Online was German terms listed in Table 1 describing pedia relies on its community to recognize judged to be superior to Wikipedia with re- thermodynamic concepts or quantities, a and correct factual errors and to improve spect to coverage and most of the formal Wikipedia entry appears as the first hit for overall style and content of the articles. and structural features, the quality assess- all of them. Hence, students who carry out Smaller errors and apparent vandalism ment by experts from ETH Zürich and the Google searches to find information that are rectified quickly, whereas errors intro- Max Planck Society favored Wikipedia ar- assists them in their assignments will come duced in the first edit may survive for quite ticles over those of Römpp Online. How- across Wikipedia articles very often. This, a while before they are detected.[2] ever, the results of the expert assessment Several studies have contrasted Wikipe- must be taken as non-representative, since dia with traditional online and print univer- only about half of the 32 survey forms sal encyclopedias such as the Britannica,[3a] were returned. Furthermore, each article Encarta,[4] and Brockhaus.[4,5a,6] Two of was rated only once, and the disciplines them concluded that the quality of selected within chemistry and pharmaceutical sci- *Correspondence: Dr. M. P. Brändlea Wikipedia articles is nearly equivalent for ences were covered rather unevenly. Tel.: +41 44 632 29 48 science articles[3a] or even better for popular We took the concept of this study as a Fax: +41 44 633 12 87 science articles[6] than their counterparts. starting point to repeat the assessment of E-mail: [email protected] aChemistry Biology Pharmacy Information Center Notably the comparison with Britannica Römpp Online and Wikipedia with a focus ETH Zürich published in Nature gave rise to some con- on a single discipline only and carried out Wolfgang-Pauli-Str. 10 troversy.[3b] In specialized scientific disci- by first-year university students. As disci- CH-8093 Zürich bLaboratory of Physical Chemistry plines, only a few evaluations of Wikipedia pline we chose chemical thermodynamics, ETH Zürich versus reference works have been done, e.g. because this topic is part of every chemis- 310 CHIMIA 2010, 64, No. 5 Hot Topics

Table 1. Alphabetic list of examined headwords in thermodynamics an institutional subscription. Articles are German headword English translation authored by about 180 paid experts from academia, industry and the government Absolute Temperatur Absolute temperature in their respective fields. Authorship is Absoluter Nullpunkt Absolute zero credited in the article header if the author Adiabate Adiabatic curve agrees. An editorial board consisting of Ausdehnungskoeffizient Thermal expansion coefficient 20 editors and the Römpp editorial office Avogadrosches Gesetz Avogadro’s hypothesis propose new encyclopedia entries and are Carnotscher Kreisprozess Carnot cycle also responsible for quality assurance. As Clausius-Clapeyron-Gleichung Clausius-Clapeyron equation of November 2007, Römpp Online con- Dampf Vapor tained about 57’000 headwords, of which Dampfdruck Vapor pressure 39% were redirects.[9] Druck Pressure The universal encyclopedia Wikipe- Enthalpie Enthalpy dia, founded 2001 by Jimmy Wales and Entropie Entropy funded by donations, is now the largest Gibbs-Energie Gibbs energy encyclopedia worldwide with more than Gibbssche Phasenregel Phase rule 15 million articles, created by the collab- Hessscher Satz Hess law orative work of about 91’000 active con- Isobare Isobar tributors in more than 270 languages (as Isotherme Isotherm of January 2010).[19] The German Wiki- Kalorimetrie Calorimetry pedia[20] with about 1.04 million articles Kompressibilität Compressibility is the second largest after the English Kritischer Punkt Critical point edition. There is no editorial board or of- Normalbedingungen Standard conditions fice as with Römpp. However, registered Partialdruck Partial pressure users who publish many articles may ad- Phasendiagramm Phase diagram vance to members with higher administra- Reaktionsenthalpie Reaction enthalpy tive rights. For the German Wikipedia, a [21] Schmelzwärme Heat of fusion chemistry panel of about 30 members Spezifische Wärmekapazität Specific heat (capacity) who develop guidelines and style rules for chemistry articles[22] has been formed. Tripelpunkt Triple point Half of its members are chemistry en- Verdampfungswärme Heat of vaporization thusiasts, the other half are professional Volumenarbeit Pressure-volume work chemists including master students, PhD Wärme Heat students, PhDs and even a professor of chemistry. As of November 2007, the German Wikipedia covered about 29’000 headwords in the field of chemistry, of try curriculum, well covered in textbooks, the encyclopedia articles are suited for. which 42% were redirects.[9] highly formalized and comes with a rich Only a few papers[12–14] or projects[15] – and often confusing – terminology. Our exist that actively promote teaching with study had several objectives. As a didac- Wikipedia and other online encyclopedias, Method tic measure, we set up the assessment as a which shows that instructors are still reluc- written assignment during the thermody- tant about the use of these new media. We Comparison of German Wikipedia and namics course. We expected from this that hope that by our contribution we can en- Römpp Online was carried out as one of students would actively apply and learn the courage other instructors to develop ideas twelve written assignments of the second subject matter and become better prepared on how to incorporate online encyclope- semester thermodynamics course at ETH for the exam by reading other sources in dias in the classroom. Zürich. This course is attended by students addition to the lecture manuscript and the of chemistry, chemical engineering, and recommended textbooks.[10,11] By provid- molecular biology. Participation in this ing the students with different assessment Römpp Online and Wikipedia survey was non-mandatory, since only criteria, we hoped that they would learn ten of the written assignments had to be to critically judge the quality of reference The Römpp Chemielexikon was found- completed to obtain the credit points. The works and improve their information lit- ed in 1947 by chemistry teacher Hermann survey was handed out as one of the last eracy. Furthermore, we expected to obtain Römpp and is now published by Thieme. assignments in the semester, so that it was recommendations for use of the chemistry The 10th and last print edition dates 1996– safe to assume that the students already encyclopedias as instructional materials in 1999 and covers six volumes on chemis- had acquired firm knowledge to respond in chemistry lectures and courses in chemical try and five additional volumes on bio- a competent manner. For the comparison, a information. Also, we wanted to know to chemistry and molecular biology, paints Web-based survey form was installed. The what extent the two encyclopedias are able and printing inks, food chemistry, natural survey form was available for a four-week to support the learning process and fulfill products, and environmental science. It is period (23.4.–17.5.2009). Thirty lemmas the requirements of being an introductory considered the standard reference work (see Table 1) of thermodynamics topics source. Students and experts may look at in German-speaking university chemistry taught in the lecture were available for rat- scientific texts in a different way: Which libraries.[16,17] The eleven volumes were ing. These were selected from a database- criteria do chemistry students regard as rel- converted in 2002 to the Web-based en- generated list of lemmas[9] in physical evant to assess the quality and utility of an cyclopedia Römpp Online,[18] which is chemistry covered by both encyclopedias. encyclopedia article? Finally, we wanted to updated semi-annually. The online ver- After a student had selected a headword learn from the students, which target groups sion can only be accessed on the basis of from the list, links to the correspond- Hot Topics CHIMIA 2010, 64, No. 5 311

ing encyclopedia articles were presented. Table 2. Average marks m for Römpp Online and Wikipedia, calculated by averaging the criterion Students had to rate at least three lemmas marks over all articles, number b of articles that were rated bettera, and standard deviation s. from both encyclopedias (or six articles), Mark scale ranging from 1 (bad) to 5 (excellent). The superior values of m and b are indicated by but were free to select the topics from the bold printing. list. This indicated to us which topics were Criterion Römpp Online Wikipedia preferred by the students. To check who m b s m b s completed the survey, the student had to Accuracy 4.44 16 0.78 4.34 8 0.72 submit his/her student ID. The survey contained four criteria to be Completeness 3.43 6 1.14 4.17 22 0.82 rated with regard to content and four crite- Length 2.99 4 1.22 3.91 24 0.96 ria with regard to the form of the articles. Comprehensibility 3.80 5 1.04 4.29 22 0.73 The same criteria were used as in the survey of experts carried out in 2008.[9] Crite- Sources 3.64 14 0.93 3.42 10 1.12 ria with regard to content were accuracy, Further reading 3.56 9 1.01 3.90 16 0.88 completeness, length, and comprehensi- Web links 3.22 3 1.01 4.01 24 0.91 bility. Two additional text fields allowed comments on mistakes and omissions in Images and graphics 3.15 2 1.10 3.96 25 1.05 the selected article. Criteria with regard Average of criteria marksb 3.61 4 1.14 4.07 21 0.91 to form were availability and quality of Average overall markc 3.23 6 0.96 3.95 23 0.79 literature sources the article is based on, a references for further reading, links to ex- Counted as better if the absolute value of the difference between marks was greater than 0.1, otherwise counted as equivalent; bCalculated as arithmetic mean over the marks for all criteria ternal websites, and images and graphics, except the overall mark; cAverage over overall marks supplied by students respectively. For these criteria, it was possible to assign a mark only after one had checked a tick mark to indicate that corresponding items exist in the article. In addition, an overall mark had to be attributed words were ‘Spezifische Wärmekapazität’, ergie’ (2.17, 18 ratings), ‘Reaktionsenthal- to the article. The available marks ranged rated twice, along with ‘Kalorimetrie’ and pie’ (2.00, 5 ratings), and ‘Ausdehnungs from 1 (bad), 2 (insufficient), 3 (sufficient), ‘Avogadrosches Gesetz’, which were both koeffizient’ (1.71, 7 ratings). The three 4 (good), to 5 (excellent). Finally, the sur- rated only once. worst-rated articles contained – at the time vey asked the student to select one or more Table 2 compares the average marks of when the survey was carried out – only the target groups that the article may be suited the eight criteria used for rating of the ar- minimum information by stating the defi- for. As target groups ‘pupils of secondary ticles. Overall, Wikipedia obtained better nition of the respective quantity in one or school level 2’, ‘students’, ‘PhD students’, marks than Römpp Online, with the excep- two sentences only, which was criticized ‘PhDs and experts’, and ‘general public’ tion of the criteria accuracy and sources. in the submitted comments. For Wiki- could be chosen from a list. For each of This trend is also reflected in the number of pedia, the three articles ‘Verdampfungs the eight criteria and the target group, an articles that were rated better in one ency- wärme’ (4.67, 3 ratings), ‘Normalbedin- additional text field allowed comments on clopedia compared to the other. For all cri- gungen’ (4.64, 11 ratings), and ‘Ausdeh- the rating or the selection. teria, the average mark over all headwords nungskoeffizient’ (4.57, 7 ratings) were To stimulate the students’ interest and was at least ‘sufficient’ (3 or greater) for ranked best, whereas ‘Isotherme’ (3.38, commitment for the assignment, a 15 both encyclopedias. 16 ratings), ‘Isobare’ (2.71, 14 ratings), minute kick-off session was held during Fig. 1 and Fig. 2 compare the average and ‘Kalorimetrie’ (2, 1 rating) obtained a lecture, giving more background and overall marks attributed to the individual the lowest overall marks. While the article information on the purpose of the survey. articles of both encyclopedias. In Fig. 1, ‘Isotherme’ was rated as sufficient and The final results and the evaluation of the the headwords on the abscissa are sorted also obtained favorable comments, ‘Iso- survey were then presented during the final by decreasing average overall mark of the bare’ obtained insufficient marks with re- lecture of the thermodynamics course. Römpp Online articles and in Fig. 2 of gard to completeness, length, and sources. the Wikipedia articles, respectively. The In the comments to ‘Isobare’, the students average overall marks for Römpp On- often criticized that the article referred to Results line articles are lower than the ones for meteorology rather than to thermodynam- the Wikipedia articles, ranging from 1.7 ics; seven comments mentioned omissions 215 students registered for the thermo- to 4.1 compared to 2.0–4.7, respectively. such as heat and entropy change associ- dynamics course, so that in theory 1290 For 23 headwords (77%), the Wikipedia ated with an isobaric process which are ratings could be expected. Of these, 99 stu- articles obtained a higher mark than the described in the corresponding Römpp dents (46%) contributed 619 ratings (310 corresponding Römpp Online articles. One Online article. The single rating on article for Römpp Online and 309 for Wikipedia), lemma was ranked equally, and six Römpp ‘Kalorimetrie’ gave insufficient marks (2) which corresponds to a feedback rate of Online articles were ranked better than the for length and sources, but a mark 3 for ac- 48%. 91 students (42%) submitted five or corresponding Wikipedia article. Seven ar- curacy and comprehensibility. In the com- more ratings. One diligent student even ticles in Römpp Online and two articles in ment, the student mentioned that the topic contributed as many as 11 ratings. Wikipedia obtained an insufficient average was “not well explained at all”. The favorite headwords were ‘Absolut- overall mark (below 3). To check whether the overall marks re- er Nullpunkt’, rated 42 times for Römpp For Römpp Online, the top three over- flect the impression the participants gained and 38 times for Wikipedia, followed by all marks were given to articles ‘Druck’ from their rating of the individual criteria, ‘Absolute Temperatur’, rated 19 times for (average mark 4.10, 10 ratings), ‘Gibbs we calculated an average from all marks Römpp and 20 times for Wikipedia, and sche Phasenregel’ (4.08, 12 ratings), and that were attributed to the criteria. The de- ‘Gibbs-Energie’, rated 18 times for both ‘Kalorimetrie’ (4, 1 rating). Articles with viation between the averaged marks over encyclopedias. The most unpopular head- the worst overall marks were ‘Gibbs-En- criteria and the students’ overall marks, 312 CHIMIA 2010, 64, No. 5 Hot Topics

Average Overall Mark Average Overall Mark 1 2 3 4 5 1 2 3 4 5

Druck Verdampfungswärme Gibbssche Phasenregel Normalbedingungen Kalorimetrie Ausdehnungskoeffizient Isobare Spezifische Wärmekapazität Hessscher Satz kritischer Punkt Wärme Entropie Enthalpie Partialdruck Absolute Temperatur Volumenarbeit Carnotscher Kreisprozess Clausius-Clapeyron-Gleichung Tripelpunkt Gibbs-Energie Entropie Phasendiagramm Dampf Adiabate Clausius-Clapeyron-Gleichung Absoluter Nullpunkt Verdampfungswärme Avogadrosches Gesetz Isotherme Tripelpunkt Kompressibilität Carnotscher Kreisprozess Volumenarbeit Absolute Temperatur Phasendiagramm Enthalpie Absoluter Nullpunkt Druck Adiabate Kompressibilität Dampfdruck Dampf Avogadrosches Gesetz Dampfdruck Spezifische Wärmekapazität Gibbssche Phasenregel Partialdruck Hessscher Satz kritischer Punkt Reaktionsenthalpie Schmelzwärme Schmelzwärme Normalbedingungen Wärme Gibbs-Energie Isotherme Reaktionsenthalpie Isobare Ausdehnungskoeffizient Kalorimetrie

Fig. 1. Average overall marks given for individual headwords, sorted Fig. 2. Average overall marks given for individual headwords, sorted by decreasing value for articles in Römpp Online. Continuous line with by decreasing values for articles in Wikipedia. Continuous line with triangles: Römpp Online, dashed line with squares: Wikipedia. Mark triangles: Römpp Online, dashed line with squares: Wikipedia. Mark scale ranging from 1 (bad) to 5 (excellent). scale ranging from 1 (bad) to 5 (excellent). both listed in Table 2, is small. The latter ria were relevant and other, unconsidered length and completeness for Römpp On- are only slightly lower by 0.38 for Römpp parameters such as style or conciseness line (0.70), which will be discussed below. Online and 0.12 for Wikipedia. Average would not have a significant influence on For both encyclopedias we wanted to marks over criteria and overall marks for the overall assessment. find out to what extent the marks of the individual articles correlate well for both Table 3 displays correlation coeffi- eight criteria enter into the overall mark. encyclopedias. For Römpp, the correla- cients between ratings of different crite- We assume that an overall mark oj of a rat- tion coefficient is 0.77 and for Wikipedia ria with regard to content. There is only ing j can be described by the linear equa- 0.76, which indicates that the chosen crite- one strong correlation, namely between tion

8 Table 3. Correlation coefficients between marks regarding content criteria. 0 1 ∑wicij −o j = i=1 (1) Completeness Length Comprehensibility

Römpp Wikipedia Römpp Wikipedia Römpp Wikipedia with wi = weight and cij = jth mark for criterion i, oj = overall mark,2 and i running over all criteria.n  8 Since 310 ratings were ob- Accuracy 0.26 0.50 0.14 0.34 0.28 0.42 2 tainedR = for∑ Römpp∑wic ijOnline− o j  articles and 309 j =1 i=1  Completeness 0.70 0.48 0.16 0.32 ratings for Wikipedia articles, the system of linear equations is overdetermined, but Length 0.22 0.31 optimum weights can be found by mini- mizing the sum of the squares 8 0 1 ∑wicij −o j = Hoti=1 Topics CHIMIA 2010, 64, No. 5 313

2 n  8  Fig. 3. Distribution of 2 100% target groups named R = ∑∑wicij − o j  j =1 i=1  (2) by students for 80% encyclopedia articles. for the n rated articles of the respective en- Multiple selections were allowed. Light 60% cyclopedia. The optimization was carried bars: Römpp Online, out with the Microsoft Office Excel add-on dark bars: Wikipedia. Solver which uses the generalized reduced 40% gradient algorithm. For different sets of 20% initial guess values for wi, always the same optimized values resulted after each run. The weights found for Römpp Online and 0% Wikipedia (values in brackets) were 0.42 General public Pupils of Students PhD students PhDs and secondary experts (0.36) for completeness, 0.24 (0.24) for school level 2 length, 0.14 (0.10) for comprehensibility, 0.07 (0.19) for accuracy, 0.05 (0.06) for sources, 0.05 (0.03) for images and graphics, 0.03 (0.04) for further reading, and Table 4. Student participation in survey and exam. Values indicate number of students. 0.00 (0.01) for links. Note that the weights Survey Exam Total are not normed – in the case of Wikipedia they add up to 1.03 – since the only bound- passed failed not adhered ary condition applied in the optimization completed 35 49 7 91 was wi ≥ 0. The R value for Römpp Online is 86, for Wikipedia 67.9. not completed 51 46 27 124 Fig. 3 shows the distribution of target groups for which the survey participants Total 86 95 34 215 indicated articles are suited for. Note, since multiple selections per rated article were possible, the sum of the percent- ages exceeds 100%. Both encyclopedias Therefore, one should have in mind to tained 119 (39%) comments on length, were rated adequate for students as target prepare one or two knowledge requests in of which 48 (40%) stated that the article group (Römpp 73%, Wikipedia 91%), with case of a possible repetition of this survey was “too long” or “too elaborate”, whereas Römpp Online catering more to the expert in another discipline. One has to note also only 17 (14%) comments mentioned that it and Wikipedia more to the general public. that this assignment was one among eleven was too short. Overall, the large number of Table 4 lists the participation of stu- other written assignments which covered comments indicates that the students took dents in the survey and in the course exam, problem solving questions and therefore the written assignment seriously; also they as well as the number of students who were made to prepare for the exam. expressed strong interest in the results pre- passed the exam. Of the 215 students who The largest Wikipedia-Römpp differ- sented by us in the last lecture. registered to the course, 181 (84%) took ences between the marks in Table 2 can be Both encyclopedias obtained very good the exam. 86 of these (47.5%) passed the found for the criteria length (0.92), images average marks for the accuracy of their ar- exam. Of the 84 students who took the and graphics (0.81), Web links (0.79), and ticles. The difference between them is very exam and also completed the survey, 35 completeness (0.74). However, accord- small (0.10), which is also supported by the (42%) passed and 49 failed (58%). On the ing to the results of our fit of weights fact that only a few mistakes were reported, other side, 51 (53%) students of the 97 who described above, students attribute little namely 15 factual errors for Römpp Online did not participate in the survey passed the significance to formal criteria such as im- versus 18 for Wikipedia. If one adds also exam, but 46 (47%) failed. Given that the ages and graphics and web links, but high the factual errors reported in the comments off-diagonal numbers (51 passed/not com- importance to the content criteria com- for accuracy, one finds 25 factual errors pleted and 49 failed/completed) are larger pleteness and length. This is especially for Römpp Online and 26 for Wikipedia than the diagonal values (35 passed/com- true for Römpp Online, for which we find (or for both in 8% of the ratings). pleted and 46 failed/not completed), one that marks regarding completeness and We found that some students were not has to conclude that participation in the length are correlated significantly. That familiar with the concepts sources, further survey did not have any effect on success length and completeness of the articles reading, weblinks, images and graph- in the exam. This finding is also supported were key parameters for the students can ics, since we observed for several articles by the small, negative correlation coeffi- also be seen from the number of comments that for example existence of images and cient of –0.11. they had submitted with their ratings. For graphics was checked with a tickmark, Römpp Online, 142 (46%) comments on although there were none in the original length were obtained. Two thirds (94) of article. Apparently, some took the math- Discussion these mentioned that the article was “too ematical formulas in the articles for graph- short” or consisted of “only one sentence”, ics. Also, it was not clear to students how to The relationship between participation but only five (4%) said that it was too long. treat web links, since internal links to other in the survey and success in the exam may This trend is also reflected in the number encyclopedia articles were counted as well come as a surprise. The main reason for of comments on omissions, for which 110 as links to external webpages, although we this result is that the exam questions were (35%) comments were returned for Römpp had expected that only the latter should be not designed to test the knowledge which Online and only 38 (12%) for Wikipedia, counted. Similarly, the meaning of sourc- was imparted by the encyclopedia articles but less so in the number of comments on es and further reading was intermingled. – definitions, examples, applications – completeness (89 Römpp versus 70 Wiki- Therefore, we counted the existence of an but were problem solving questions only. pedia). Otherwise, Wikipedia articles ob- item only if it was ticked by at least half 314 CHIMIA 2010, 64, No. 5 Hot Topics

of the participants in their ratings. For In our chemical information courses objections, Nature’s responses to Encyclopedia the 30 articles, we found about equal use for bachelor students, we promote Römpp Britannica and supplementary material available on http://www.nature.com/nature/ of sources (17 versus 19 as indicated by Online and other commercial encyclope- britannica/index.html (28.3.2010). students) and further reading (both 16) in dias as an entrance to quickly obtain an [4] C. Schlieker, ‘Wissen auf wikipedia.org: both Wikipedia and Römpp Online, but overview, relevant sources and further Explorative Untersuchung von Wissen in much higher use of web links (21 versus 4) literature. In many chemistry disciplines, kollektiven Hypertexten’, Version 1.01, and images and graphics (20 versus 5) in Diplomarbeit, Univ. Bremen, 2005; http:// Römpp Online does well for that purpose. www-user.uni-bremen.de/~chrof/wissen_auf_ Wikipedia than in Römpp Online. However, to be more attractive to students, wikipedia.pdf (7.4.2010). This is obviously related to the fact articles should be written with more focus [5] a) R. Hammwöhner, K.-P. Fuchs, M. Katten that, unlike Römpp Online, Wikipedia on didactics. beck, C. Sax, in ‘Open Innovation’, Ed. was designed as a Web medium from the Given the survey results, it is remark- A. Osswald, M. Stempfhuber, C. Wolff, Proc. 10. Internationales Symposium für very beginning. Römpp, as a former print able that students do not pay more atten- Informationswissenschaft, Konstanz, 2007; medium, had to be considerate on space tion to number, type and quality of sources b) R. Hammwöhner, in ‘Wikis: Diskurse, and therefore works with the minimum cited in encyclopedia articles, and have a Theorie und Anwendungen’, Ed. C. Stegbauer, of graphical materials. This view is also fuzzy understanding of their importance J. Schmidt, K. Schönberger, kommunikation@ supported by the results of Sonnenstuhl,[9] gesellschaft 2007, 8, Beitrag 3; http://www.soz. to support and validate the statements uni-frankfurt.de/K.G/B3_2007_Hammwoehner. whereas, contrary to the results for the ther- made. This is especially important in the pdf (7.4.2010). modynamic article set, she found for a set case of Wikipedia where anyone can edit [6] H. Güntheroth, U. Schönert, Stern: das of 171 articles in 15 chemistry disciplines articles. Frequent use of Wikipedia articles deutsche Magazin 2007, 50, 31; http://www. that Römpp Online cites more sources and by students is a fact; students do trust this stern.de/digital/online/wikipedia-wissen-fuer- alle-606048.html (28.3.2010). further reading than Wikipedia. information. Based on the findings of the [7] L. H. Rector, Ref. Serv. Rev. 2008, 36, 7; doi: The distribution of target groups in Fig. present and the previous[9] assessment, 10.1108/00907320810851998 3 shows its maximum for the target group both Wikipedia and Römpp Online can [8] K. A. Clauson, H. H. Polen, M. N. Kamel of students, for which both Wikipedia and be recommended as a starting reference. Boulos, J. H. Dzenowagis, Ann. Pharmacother. Römpp Online are suited. In fact, it was However, chemistry lecturers, lab course 2008, 42, 1814; doi: 10.1345/aph.1L474 [9] J. Sonnenstuhl, ‘Römpp Online versus difficult for the participants to estimate assistants and chemical information in- Wikipedia: ein formaler und inhaltlicher from their level of experience what knowl- structors should insist that students do not Vergleich zwischen einer lizenzpflichtigen und edge PhD students, PhDs and experts had stop at encyclopedia texts, but use and cite einer freien Online-Enzyklopädie’, Magister acquired and would require and what type the primary literature and secondary refer- Thesis, Humboldt University, Berlin, 2008. [10] P. W. Atkins, J. de Paula, ‘Physikalische of encyclopedia content would therefore ences (handbooks, databases) in their re- Chemie’, 4. Aufl., Wiley-VCH, Weinheim, be adequate for those target groups. Hence, ports, so that students become acquainted 2006. only the statements on the target groups with the system of peer-reviewed chemis- [11] K. Denbigh, ‘The Principles of Chemical general public, pupils and students are try literature as early as possible. Equilibrium: With Applications in Chemistry significant, whereas the higher suitability and Chemical Engineering’, 4th ed., Cambridge University Press, Cambridge, 1981. of Römpp Online for PhD students, PhDs Acknowledgements [12] P. Konieczny, Int. J. Instructional Technol. and experts must be interpreted in the We thank Jana Sonnenstuhl, Berlin, for Distance Learn. 2007, 4, 15; http://www.itdl. manner that students estimate the articles providing us with the questionnaire used in org/Journal/Jan_07/article02.htm (26.3.2010). of Römpp Online as more complex and the expert assessment. Dr. Stephan Ballenweg, [13] D. Murley, Law Libr. J. 2008, 100, 593; less comprehensible than those of Wiki- Dr. Guido Hermann, and Klaus Köberlein http://www.aallnet.org/products/pub_llj_ from Thieme Verlag, Stuttgart, as well as Arne v100n03/2008-29.pdf (27.3.2010). pedia. Klempert and Daniel Kinzler from Wikimedia [14] K. Wannemacher, in ‘Advances in Web Based Deutschland are credited for providing us tables Learning – ICWL 2009’, Ed. M. Spaniol, Q. Li, of Römpp Online and Wikipedia headwords. R. Klamma, R. W. H. Lau, Lect. Notes Comput. Conclusions Sci. 2009, 5686, 434; doi: 10.1007/978-3-642- Dr. Engelbert Zass, ETH Zürich, is acknowl- 03426-8_52 edged for helpful comments. This study was [15] Wikipedia, Wikipedia:School and university In the eyes of first-year university stu- financed by the Chemistry Biology Pharmacy projects, http://en.wikipedia.org/wiki/Wikipedia: dents, both encyclopedias obtained very Information Center, ETH Zürich. SUP (7.4.2010). good ratings regarding their accuracy. [16] M. Mücke, ‘Die Chemische Literatur: Ihre Supplementary material Erschliessung und Benutzung’, Verlag Chemie, However, in the overall rating, Römpp Weinheim, 1982. Online became victim of its terse and con- The master evaluation, the detailed evaluations and comments made by students on the in- [17] R. Keese, ‘Grundoperationen der präparativen cise scientific writing style, because in organischen Chemie: Eine Einführung’, 6. dividual encyclopedia articles can be obtained Aufl., Univ. Bern, 2003. their learning process, students attach im- on request from the corresponding author. portance rather to completeness and com- [18] Thieme Chemistry – Römpp, http://www. roempp.com/ (25.3.2010). prehensibility than to exactness and accu- Received: April 7, 2010 [19] Wikipedia, Help:About, http://en.wikipedia. racy. Neither Römpp nor Wikipedia were org/wiki/Help:About (29.3.2010). originally designed with the student in [1] R. Rosenzweig, J. Am. Hist. 2006, 93, 117; [20] Wikipedia – Die freie Enzyklopädie, http:// mind and to support teaching and learning. http://www.historycooperative.org/journals/ de.wikipedia.org/ (1.4.2010). The open concept of Wikipedia attracted jah/93.1/rosenzweig.html (28.3.2010). [21] Wikipedia, Wikipedia:Redaktion Chemie, http:// [2] B. Luyt, T. C. H. Aaron, L. H. Thian, C. K. de.wikipedia.org/wiki/Wikipedia:Redaktion_ with time authors such as chemistry stu- Hong, J. Am. Soc. Inf. Sci. Technol. 2008, 59, Chemie (25.3.2010). dents and professional chemists who like 318; doi: 10.1002/asi.20755 [22] Wikipedia, Wikipedia:Richtlinien Chemie, http:// to impart their knowledge to others. Both [3] a) J. Giles, Nature 2005, 438(7070), 900; doi: de.wikipedia.org/wiki/Wikipedia:Richtlinien_ encyclopedias are now written by peers for 10.1038/438900a; b) Links to Britannica’s Chemie (25.3.2010). peers, Wikipedia mostly by non-experts for students, scholars, and the general public, Römpp Online by experts for experts. Wikipedia articles are easier to under- stand because their content and language is more textbook-like.