70-20,925 |

BERTRAM, Sheila Joan Kelley, 1940- *' THE RELATIONSHIP BETWEEN INTRA-DOCUMENT J CITATION LOCATION AND CITATION LEVEL. ]

University of Illinois at Urbana-Champaign, Ph.D., 1970 Library Science

I University Microfilms, A XEROX Company, Ann Arbor, Michigan

THIS DISSERTATION HAS BEEN MICROFILMED EXACTLY AS RECEIVED THE RELATIONSHIP BETWEEN INTRA-DOCUMENT CITATION LOCATION AND CITATION LEVEL

By

SHEILA JOAN KELLEY BERTRAM B.Sc, McMaster University, I962 B.L.S., University of Toronto, I963 M.S., University of Illinois, I966

THESIS

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Library Science in the Graduate College of the University of Illinois, 1970

Urbana, Illinois •«"M ww h T *c«uni«

UNIVERSITY OF ILLINOIS

THE CTOADUATE COLLEGE

January, 1970

I HEREBY RECOMMEND THAT THE THESIS PREPARED UNDER MY

SUPERVISION BY SHEILA JOAN KELLEY BERTRAM

ENTITLED THE RELATIONSHIP BETWEEN TNTRA-T)OnTTMF.NT

CITATION LOCATION AND CITATION LEVEL

BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR

THE DEGREE m=r DOCTOR OF PHILOSOPHY

li&jJ&e*Jr &49&LL,&^ ^ (l^fup^^^L^ Head of Department

Recommendation concurred inf

fyi. GwUo Committee .T£lo&jV> r<\\ AA*-****^*^ on r ~\n.cuajau& ^. oSU^JAXJUUC* Final Examinationf

f Required for doctor's degree but not for master's

DS17 PLEASE NOTE:

Some pages have small and indistinct type. Filmed as received,

University Microfilms iii

To Edward

Ortho and Para iv

ACKNOWLEDGEMENT

The author would like to thank the members of her * doctoral committee, Dr. Dewey Carroll, Dr. Herbert Goldhor,

Dr. Robert Coates, and Dr. John Baldwin. A special note of thanks is given to Mrs. Katherine Henderson for replacing

Dr. Baldwin at the last minute and to Mrs. Frances B.

Jenkins who provided invaluable assistance and encouragement whenever it was most needed.

Financial assistance from the University of Illinois in the form of a Higher Education Act, Title II, Fellowship in Librarianship and teaching assistantships is gratefully acknowledged.

Finally, a very special thank you to Eddy for his help, encouragement, understanding and love, which make all things possible. V

TABLE OF CONTENTS

Page INTRODUCTION 1 Definitions 3 Citation Production . 6 Citation Use 8 Hypothesis ...... hU PROCEDURE 66 Selection of Citations 66 Citation Level 99 RESULTS AND DISCUSSION 108 Frequency Distribution. 109 Citation Level Distribution 119 Self Citation and same Journal Citation Distribution , 132 Journal Distribution...... Ih2 Time Distribution 173 SUMMARY AND CONCLUSIONS 197 Summary ...... 197

Conclusions 209 Suggestions for Further Research 21j? BIBLIOGRAPHY 220 VITA 230 1

INTRODUCTION

The phenomenon of the citation, the referring from one

document to another for whatever reason, has been exploited

for many purposes. The two main categories of citation use

are the citation-tool, where the citations in a document known

to be of interest and used to find earlier cited documents

which might also be of interest (backward search) or where

the citation of a document known to be of interest is used

to find later citing documents which might also be of interest

(forward search) and the citation-count where the number of citations in some category (such as journal title, year,

language) is used as a measure either of the properties of

the citations themselves or for some other defined purpose

(often related to literature use).

The citation-tool use is very much related to the problem of relevance since the citation leads to a document which then must be judged relevant or non-relevant to the search problem. A document of interest to different search problems will have citations which are not all relevant to all of the different search problems. Thus the relevance of the cited document to the citing source document is not fixed, but will vary with the defined search problem. 2

Similarly with the citation-count use, each citation

is usually given an equal weighting with every other

citation (or not counted at all), that is the weight of each citation in the count is either one or zero. Again, it would appear that all of the citations might not have the same weight with respect to the defined purpose of the citation-count.

However, as Kaplan has pointed out, very little is actually known about the history and norms of citation 1 practices. Time and money are being spent on exploiting the citation but very little effort is being made to further understand the factors operating in the production of the citation.

In this study, it is suggested that the level (or amount) of material from a cited document that is actually being cited by the source document may vary, but in some pre­ dictable way, with respect to the location of the citation within the source document. That is, knowing the section

(introduction, experimental, results/discussion) of the source document in which the citation occurs will indicate the level of material most likely to have been cited. The specific hypothesis to be tested is that citations in the introduction section of the source document will tend to be 3

to the whole of the cited document, citations in the results/ discussion section will cond to be to a part of the cited document, and citations in the experimental section will tend to be to word/words from the cited document.

The reason suggested for the existence of such a relation­ ship is that properties of citations in general tend to vary with the section of the source document in which the citation occurs and the citation level is a measurable property of citations. It is further suggested that these three sections can then be related to the stages of research (planning, performing, analyzing) which have different literature needs, and therefore citations in the three sections will have dif­ ferent properties.

Definitions

Before proceeding, the terms footnote, reference, and citation need to be defined since they are often used inter­ changeably, inconsistently, and ambiguously. These definitions will then be used in this study (except far direct quotes) regardless of the terminology and corresponding meaning which may have been used in the literature.

citation; the indication given in the document (usually by means of a superscript or the last name of the author plus the date of publication) which leads to the specific footnote 4 which should be consulted for further information. Citation is also used as the more inclusive term indicating the whole sequence involved in the referring by one document to another document.

footnote: the physical entity, the group of words located at the bottom of the page or in a list at the end of the document.

reference; the bibliographic or non-bibliographic informa­ tion contained in the footnote. (Text is defined in a similar manner to mean the information contained in the document, while the document is the physical item.)

bibliographic citation; the bibliographic information

(in this case given in the footnote) which leads to the specific document which is to be consulted for further informa­ tion.

Document Footnote _ Document i / i Text / Reference I Bibliographic Citation Citation Bibliographic Citation 5

It is in no way suggested that these definitions are

the best, or that they should be adopted universally; however, they are adequate for this study.

A citation can lead to a footnote which does not contain

a bibliographic citation. In addition, in the literature of some scientific fields, when the same footnote is to be con­

sulted several times, a new footnote is not made each time, rather the-citation (i.e., the superscript or the name plus

date) is repeated. Therefore, the total number of citations will not necessarily equal the number of bibliographic cita­ tions. These additional citations to a unique footnote, and therefore to the same bibliographic citation, are termed repeat citations and, throughout the study, the number of unique citations and the number of gross citations (the unique citations plus the repeat citations) are investigated separately.

This is not to be considered a closed system since biblio­ graphic citations can come from many places other than foot­ notes and can in turn lead to documents which do not contain footnotes. From these definitions, a citation-count is differentiated from the larger category of a bibliographic citation-count which includes the counting of bibliographic citations found in places other than footnotes such as bibliographies, indexes, and abstracts. As Stevens has 6

pointed out, these are two different types of studies of the

literature. The bibliographic citation-count is directed

towards literature productivity while the citation-count is

directed towards literature use. Therefore the data are o obtained from different points in the communication cycle.

Citation Production

The literature on the production of citations is very

meager indeed and deals primarily with form rather than

practice. There are several amusing articles on the subject

of citations such as the ones by Sullivan, in which the

footnotes talk back to the text and end up taking over, and

Rinelander, who gives a list of the various non-scholarly 1). reasons for making footnotes.

The various writing style guides, such as directions to

authors submitting papers to the particular journal,5 give

directions for form but give little or no guidance as to

practice. An examination of the form of footnotes and

bibliographic citations in some 200 journals has been con­

ducted by Dutta, again a study of form rather than practice.

The librarian's primary concern with citations has been with the form and the identification of the bibliographic

citation. With the increasing number of journal titles,

the use of computer printouts with their accompanying use of truncation, and the need for standardization, the CODEN for o periodical titles and the work of the USASI Z39 Subcommittee g on Periodical Title Abbreviations^ have become of major importance..

There is even less information available in the litera­ ture on the historical background of the citation. Barr asked for early examples of footnotes in the literature with an indication that a study might be forthcoming; however, npthing has appeared to date. He did receive several replies with dates back as far as 1527; however, the problem of what 11 exactly constitutes a footnote was quite evident. Barnard suggests that "the practice of quoting some previous writer is almost as old as authorship itself",1 2 while Hodgson suggests that "the need for a careful indication of the sources of information had its origins in Germany about the beginning of the nineteenth century."1 J3 Price appears to consider these two ideas as two different phenomena, a difference between the literary and the scientific citation, again a problem of definition. He states, "but it is generally evident from a long run of any scientific periodical that around 1850 there appears the familiar pattern of explicit reference to previous work" and then goes on "before that time, though footnoting is as old as scholarship itself...there is nothing like this 8 attitude toward the accretion of learning." 14 As Kaplan notes, Price does not give any evidence in the form of a 15 citation to support his views, ^

Citation Use

Citations from documents of interest have been used by researchers to locate other possible documents of interest probsbly for as long as citations themselves have existed.

Lykoudis has presented a mathematical model of such a use, a search strategy based on the use of abstracts to locate documents and then further investigation of their biblio­ graphic citations.

In her survey of user studies, under the category of

"preferred ways of finding information", Barber indicates that, from the results of the various studies covered,

"following up cited references appears to be rated the best 17 method." ' This idea has also been expressed in several reviews of citation indexing by Martyn ("We know that fol­ lowing up references cited in relevant papers is the 18 scientist's most favored method of obtaining information") and Keen ("The following up of such references has always been a fruitful and well used method of tracing earlier 19 information on a subj ect"). 9

In addition, citations have been used in various studies

in many different ways. The literature reveals examples of

the study of the properties of citations themselves (journal

distribution, time distribution, subject dispersion, language,

etc.), the study of citation indexes, and the use of citations

as an indication of some other phenomenon (spread of an idea,

quality of publication, use of the literature, etc.).

The journal distribution and the time distribution cita­

tion studies in the literature and the proposed models for

these distributions will be discussed in depth since these

two citation properties are investigated in this study with

respect to the section of the source article in which the

citation occurs. For both the journal distribution and the

time distribution, there are two major methods of plotting the

observed data, viz., the number of citations against the

journal rank (time lapse) and the cumulated number of cita­

tions against the journal rank (time lapse). Percentages rather than the absolute number of citations are sometimes used, and the journal rank (time lapse) can run in either

direction, high to low or low to high. Various models have

then been proposed to obtain calculated curves which closely

approximate the observed data. For both the journal distri­ bution and the time distribution, the proposed models des- 10

cribing the behavior of these citation properties are expo­

nential in character, that is a large number of citations

is obtained from a small number of journals or years while a

small number of citations is obtained from a large number of

journals or years.

In many of the journal distribution and time distribution

studies, the different values obtained for the constants

appearing in the models have been attributed to differences

in subject fields. These differences are often then related

to the types of activity represented by those fields, such as

pure science as opposed to applied science. As an extension

of this idea of the existence of different citation distribu­

tion patterns for different fields, this study then suggests

that different types of research activity (as represented by

the different sections of the source document) will also produce different citation distribution patterns.

An overview of the literature covering citation indexes

and other uses of citation-counts will then follow as an

indication of the activity in this area. Many advantages and disadvantages of the use of citations in both the citation-tool and citation-counts have been discussed in the literature. These are briefly covered with emphasis on the fact that little is actually known about the exact 11 relationship between the cited document and the source document and the intellectual processes which have gone into the making of the citation. This leads to the major problem of "noise", the retrieval of cited documents which are not relevant to the search problem.

Journal Distribution. The first use of citations (other than as a backward search method by an individual) is usually attributed to Gross and Gross who counted the citations appearing in the Journal of the American Chemical Society for 20 1927 and compiled a list of the most cited chemical journals.

This list was then intended to be used as an aquisitions list for the librarian, an indication of which journals should be important to libraries serving chemistry majors. Citations to the Journal of the American Chemical Society were not counted as Gross and Gross felt that this would introduce a bias.

Its choice as the source journal indicated that it was already considered to be the most important for the field of chemistry and a later supplementary count by Brown showed that the

Journal of the American Chemical Society was indeed the most cited journal.2 1

Many studies followed covering various fields, various source journals, and various years. Changes from the Gross and Gross method have been minor and have included many 12 variations of using one or more source journals, one or more years, etc., and were compiled for various reasons including acquisition policy, optimum size of journal collection, central storage, accessibility of journal materials, physical and bibliographical accessibility, and other related problems.

The culmination of this type of activity was the Brown study where the results of many of these past studies plus some op additional new ones were summarized.

These early studies on journal distribution were directed at compiling lists of the important journals in a field. How­ ever, interest in the journal distribution of citations also developed in the direction of trying to describe the observed distribution patterns by means of mathematical models. These proposed models have all been exponential in form in order to describe the rapid fall-off of the number of citations with respect to the number of cited journals.

In 1934 Bradford proposed his "law of scattering" (which went largely unnoticed until the publication of the law in his book in 1948). The law states that "if scientific journals are arranged in order of decreasing productivity of articles on a given subject, they may be divided into a nucleus of periodicals more particularly devoted to the subject, and several groups or zones containing the same number of articles 13 as the nucleus, where the numbers of periodicals in the nucleus and succeeding zones will be as l:n:n 2 ."2 J3

He then plotted the number of articles against the log of the journal rank and proposed the following model for his law: R = a log T + b where R = cumulated number of articles,

T = journal rank, a and b = constants.

Vickery later illustrated that Bradford had made an error in the derivation of the mathematical model of his law and, as a result, the model described only the upper portion of the curve. He deduced the correct formula and showed that this model would hold for more than the three zones indi- 24 cated by Bradford.

While Bradford considered bibliographic citations found in bibliographies his law has also been adopted for the journal distribution of citations as well.

Groos has noted that plotted points will eventually form a curve which falls below the Bradford predicted values, a 26 phenomenon which has been referred to as the "Groos droop."

He indicated that the larger the number of plotted points, the greater the discrepancy; therefore the size of the sample will have a definite effect on the fit of the model. Barker found that the greatest discrepancies occurred at the two ends of the distribution and suggested that the wider scatter than 14 predicted among little cited titles was a result of larger samples. Similarly, a broader definition of journal might also lead to a larger than expected number of infrequently cited titles.2'

Kendall illustrated that the "Bradford effect", as he called it, was actually an expression of the more general

Yule distribution which also operates in other fields such 29 as the productivity of authors (Lotka's law) 7 and the size of cities (Zipf's law).30

In simple terms, the highly skewed, J-shaped Yule -k distribution has the form y = ex . Barker used this Yule distribution in the form: T = a/C; in logarithmic form, log T = log a - b log C where T = journal rank, C = number of citations, a and b = constants. This is the reverse of most journal distribution models where the journal rank is the independent variable and the number of citations is the dependent variable. Because of this, Barker reversed the b -b ^1 coordinates to obtain the more common form: C = a/T (= aT" ).

Actually, Dalziel had earlier proposed a model to describe the journal distribution of citations which he used only for the first 24 ranked journals excluding same journal citations.

This simple exponential model for the apparent value of a journal was: V = aN_n™ and therefore is identical to the model used by Barker. 15 Cole plotted R/RT, the cumulated fraction of citations, against log T/TT, the cumulated fraction of cited titles

(in rank order) which is similar to Bradford's model using percentages instead of absolute numbers. He suggested that the slope a of his model (R/RT = a log T/TT + b) is "clearly a measure of the extent of scatter, and its numerical value, to which it is proposed to apply the term 'reference-scattering coefficient', may be a characteristic of the literature in any particular field." He also found that he got better- fitting calculated straight lines with data taken from reference question records and abstract counts than from citation-counts (the citation-counts also gave lower values) and so concluded "the above comparison of reference scattering coefficients indicates that the reference-count method is inadequate" and shows that the reference-count method gives

"untrustworthy" results. (A better conclusion might have been that his model was "inadequate" to explain the data obtained by the citation-count method.) This again might be related to the idea presented in this study, that the relationship between the cited articles and the source article is not a constant, while articles included in a bibliography are much more likely to be related to one specific topic. 16

Leimkuhler has proposed a more sophisticated model for

the "Bradford distribution" the inverse of Bradford's law:

Rn = ^(C10^1 + Bn/N)/(log(l + B)) where N = total number

of journals cited, n = journal rank, R and IL_ = correspond­

ing number of references, B = a constant characteristic of

the subject field. He also showed that "Bradford's law and

Zipf's law are essentially just two different ways of looking 34 at the same thing." As Barker pointed out, Dalziel's model

and Kendall's model are inversely related. -* Kendall has also shown that his model and Bradford's are mathematically

equivalent. Thus these models are all inter-related (as might be expected since they are describing the same data).

Brookes recently published an article on the derivation and application of the Bradford-Zipf distribution and pointed out that the frequency of citation to journals is not identical with the frequency of occurrence of relevant papers in the journals; "that citation data are of interest as an indication of usage rather than occurrence" a point which is missed in many comparative studies. He also points out the important idea that Bradford's law states mathematically that articles published in individual scientific periodicals tend to be 37 concentrated in specific areas. ' As a result of Brookes' 17 article, there has been some discussion in the literature as

to the history, relationship, and priority of these various 38 models.

Time Distribution. Most of the early citation-count

studies dealing with journal distributions also included

some time distribution data as well, with the journal distri­

bution being given for various time periods.

As with journal distribution, many researchers have pro­

posed exponential models for the time distribution of citations

to explain the fall-off in the number of citations with time.

Since there are more factors involved in the time distribution,

such as growth and obsolescence of the literature with time,

disruptions in literature production during war years, dif­

ferent types of cited literature with respect to time, and

the fact that the time distribution has a fixed order (years)

as opposed to the journal distribution, these models do not

appear to be as well developed or as widely accepted.

One of the first known proposed models for the time dis­

tribution is that of Gosnell who studied bibliographic cita­

tions to books from several bibliographies and proposed an

exponential model of obsolescence: y = y b where y = number of citations, x = time lapsed, b and y = constants. This

is different from the Yule type distribution of the journal 18

distribution in that the independent variable rather than a

constant appears as the exponential power. The logarithmic

form is log y = log y + x log b, and the data are therefore

plotted in semi-log form. He also proposed another measure

W (the value of y when b = 1) to indicate the rate of decay.

The various values obtained for W from the different biblio­

graphic citation-counts were then related to the various sub­

ject literatures covered in the bibliographies. While this

is not a citation time distribution study, it is worth noting 39 as an early example of a time distribution model.

One model of the time distribution for citations has been proposed by Cole: Rx = RT x e~ where R = cumulated number of citations older than x years, RT = number of citations (there­

fore a constant) and A = constant characteristic of the 40 subject field.

In an interesting analogy, Weiss compared the life span of pure data to the growth process in higher animals and suggested that death comes either as a result of assimilation or oblivion, and the less the data lend themselves to assimila­ tion, the longer the life span. From this he suggested that data which could be generalized would be assimilated faster than those which could not. He plotted the number of cita­ tions against the time elapsed for two source journals in the 19

field of biology (one which he felt had a strong analytical

emphasis, the other a largely descriptive bias) and found

that the descriptive source journal had a flatter time dis­

tribution curve (a greater dependence on older records).

Thus Weiss, in effect, is indicating that citation properties may vary with the character of the source journal. He also suggested that the ratio between the two types of data would be expected to vary with the subject field. He proposed that separate media of publication be set up in each subject area and that articles be published in one or the other media depending on the author's assessment of the prospective useful life span of the data contained in the article.4 1

Price has studied the network obtained by linking each document to the other documents directly associated with it through citations in order to indicate the nature of the research front. He identified two types of connection and concluded that "it appears that the citation network shows the existence of two different literature practices and of two different needs on the part of the scientist...(i) the research front builds on recent work, and the network becomes very tight...(ii)...a drawing upon the totality of previous work." He considers the latter to be a random process where the number of papers available for citation grows 20 exponentially at the same rate as the decrease in the chances of citation. In his study, Price indicates that virile fields as opposed to well-established fields and science as opposed to technology tend to make greatest use of the recent litera­ ture. Price has also identified variations in the normal time distribution due to the influence of the disruption of 42 scientific production during both World Wars.

This idea of two distinct types of literature use was in support of the work of Burton and Kebler who presented an analogy between the half-life of radioactive substances and the rate of obsolescence of scientific literature. The exponential model proposed for plots of the cumulated per- centage of citations against time lapse was: y = 1 - (ae + be ) where y = cumulated percentage of citations, x = time lapsed (in decades), a + b = 1. They showed that their calculated half-life varied with the subject field. Since the analogy with radioactive decay did not hold completely, they suggested that "the periodical literature of a subject field is composed of two or more distinct types of literature, each having its own half-life. There is for example in most fields, a body of literature which is referred to as the classic literature. Presumably this classic literature has 21 a relatively longer half-life than so-called ephemeral literature." 3

Meadows also found that he could divide the time curves he obtained using a citation-count into two parts, "an initial, fairly rapid exponential decay followed by another which is considerably less rapid."4 4

Barker has suggested that the time distribution of cita­ tions ought not to be directly equivalent to the measurement of obsolescence since the growth of the literature must also be taken into consideration: "the early literature is referred to less than the recent, not only because its useful­ ness has declined but also because there was less of it in the first place." In his model, unlike Price, he did not consider the rate of growth to be the same as the rate of obsolescence bx of the literature: Yc = (a/b)e - c where Yc = cumulated percentage of citations, x = time lapse (since oldest date) a and b = constants, c = (a/b)e ° where t is the source year (when the data are plotted in log-log form, c as a constant disappears). •*

In contrast to the accepted idea that the literature obsolesces with time, Dennis, in refuting the work of Lehman

(who used citations to partially support his hypothesis that 46 the most creative work is done in the early years of life), 22 used citations obtained from source books in various fields and showed that, for the period 1800-1900, older material had a better chance of being cited than later material since the probability of citation did not increase proportionately 47 with the growth of the literature. ' However, it is sug­ gested that citations from a source book are very specialized, referring usually to "firsts" and a better conclusion might have been that the number of "firsts" do not increase pro­ portionately with the growth of the literature.

MacRae recently'has also presented a model for the time distribution of citations which takes into account "not only the authors' selectivity, but also...the general growth of the relevant literature." In dealing with Price's suggestion of the two types of literature use, he presents an additional factor for the author's selectivity in favor of recent publi­ cations: C = ae' m' P where t = given year, K = factor for the growth and decay of the literature, m = selectivity factor.

He further suggests that the value of K will depend on the U8 subject field.

Of particular interest to this study are the suggestions by Burton and Kebler, Price, and MacRae that there are two types of citation patterns, the use of the recent or ephemeral literature and the use of the older or classical literature. 23

Bourne reviewed some 50 different studies covering various fields of science and technology on the use of litera­ ture as a function of age, many of which were based on citation-counts. He suggests that such time distributions can be used to determine the number of years of material re­ quired to provide a specified fraction of the users' need for literature. Plots were made of the cumulated percent of citations (as a measure of percent of users' needs satisfied) against the time tapse, from the data taken from the various studies. The half-life of the time distribution is also discussed and shown to be quite "fuzzy" within the studies of a particular field and among studies of different fields, 49 rather than specific constants. * However, in a plot for the field of chemistry, for example, he has combined data from both pure chemistry and chemical engineering citation-counts.

While no new models were proposed, this is a very useful summary of much of the work to that date.

Buckland and Woodburn have combined some of the available knowledge on scattering (journals) and obsolescence (time) of citations in order "to indicate optimal decision in the management of collections of journals." As they point out,

"the formulae used have been chosen somewhat arbitrarily but any others might have been used in the same manner." This 24 again is an oblique reference to the idea that most of the proposed models for journal distribution and for time dis­ tribution are inter-related. The formulae used were those proposed by Leimkuhler (journal distribution) and Cole (time distribution).5°

In the present study, very simple exponential models are used for the journal distribution and the time distribution of the citations since the emphasis is not on showing how well the model fits the observed data or obtaining representative constants, but to illustrate the presence of different behavior in the journal distribution and time distribution of the citations appearing in the three sections of the journal article. The two models used are (1) C = aT- , in logarithmic form log C = log a - b log T where C = number of citations,

T = journal rank (or time lapse), a and b = constants, and

(2)CC = a+b log T where CC = cumulated number of citations,

T = journal rank (or time lapse), a and b = constants.

Citation Indexes. The capability of the computer to manipulate large amounts of data has lead to the production of the Science Citation Index. There are a great many articles in the literature about this particular citation index and citation indexes in general. The availability of Science

Citation Index in turn produces further uses of citations, 25 not only as the forward citation-tool search method it is primarily intended to be, but also in citation-counts since citations from a large number of journals are brought to­ gether in one central source. For example, Martyn used the tapes from Science Citation Index I965 for a citation-count covering I965 and I964 only, in order to prepare a list of the most cited United Kingdom journals for the whole field of science (as opposed to the earlier lists confined to several source journals in specific subject areas). And of course the existence of Science Citation Index has lead to comparative studies with traditional subject indexes through the prepara­ tion of bibliographies or the retrieval of relevant citations.

These include the studies by Ghosh, Huang, and Spencer.

In addition, the preparation of citation indexes for specialized fields such as statistics (Tukey), "* new educational media (Goodman), and patents (Reisner, Garfield)"'' have been discussed in the literature. Lipetz prepared a citation index for the field of physics which leads from Physical Review and

Journal of Applied Physics articles to translated' Soviet physics journals. He then measured the impact upon physicists by measuring the subsequent utilization of these translations in an experiment to see if scientists would make use of a 58 citation index if it were available. Other examples of 26 citation indexes include the Genetics Citation Index, the citation index to the Annals of Mathematical Statistics, volumes 1-31, and the citation index published in the monthly issues of the Journal of Histochemistry and Cyto­ chemistry.

Brown has reported the use of citation indexes to check recent citations made to items in an existing bibliography 59 as a method of keeping the.bibliography up to date.

Martyn did a somewhat similar study for his review of citation indexes. Similar to updating bibliographies is the use of citations in SDI (selective dissemination of information) systems such as that described by Deweze and the commerciallaeri y 62 available ASCA, a by-product of Science Citation Index.

Very closely related to the citation index is the work which has been done by Kessler on his concept of bibliographic coupling (defined as one item of reference used by two papers, as opposed to the direct relationship between source papers and cited papers) and coupling strength (defined as the number of references other articles share in common with a test paper)x. 63

Other Uses of Citation-Counts. The first use of citation- counts other than for journal distribution and time distribu­ tion is thought to be by Sheppard who counted citations in 27 chemistry literature and decided, from the large proportion

to German journals, that chemists still needed to study 64 German. Other studies have included the use of citations

to determine subject dispersion by ascertaining the subject

classification of the journals and books cited. McAnally

(United States history) 3 and Fussier (chemistry and physics) were among the first to use this methodology. Broadus has recently done such a study with the citations in the American

Sociological Review.

Citations have been used by Dahling to trace the dif­ fusion of an idea (Shannon's theory of communication) through the literature. In addition, he plotted the network of linkages within and between fields and showed that some papers 68 became sociometric "stars" while others became "isolates."

Garfield, Sher, and Torpie did a similar study comparing the history of DNA as given in The Genetic Code by Isaac Asimov with the network produced by citations found using Science

Citation Index. There was a strong enough relationship be­ tween the two for them to suggest the use of citations as 69 a method for study in the history of science.

Another recent area of interest in citation studies is the matrix formed by journals citing journals. Xhighnesse set up a closed network or matrix so that he could study 28 reciprocal citation among twenty journals in the field of psychology and found that within the network, journals "tend to cite each other in proportion to the extent that others 70 cite them."' Other studies on such citation matrices have 71 72 been done by Garner' and Yagi.'

The number of citations to articles published by an author has been investigated as a measure of quality of publication. Westbrook used a citation-count in order to identify laboratories doing the most significant work in the field of ceramics as he felt that this would be a better measure of "natural selection" than counting published papers.'^

Sher and Garfield looked at the number of citations made in

1961 to the work of the Nobel prize winners for I962 and I963 and found that the average number of citations to their work was I69.O with an average number of 58.1 cited articles. This compared to the I96I mean of 5.5 citations per author and 74 3.4 cited articles per author.' Cole and Cole found a higher degree of correlation between the,average number of citations and other independent measures of quality than when the average 75 number of publications was used. Bayer and Folger also tested the use of citations as a measure of quality and found a low but positive correlation with the quality of graduate education. 29

Another unique use of citations (in this case non- bibliographic citations) appears in the study by Webb and

Salancik who found that studies reported in Journalism

Quarterly for 1964 which acknowledged financial support were more likely to report statistically significant tests than those not acknowledging financial support.'7'7

The use of citations to derive subject index terms has also been proposed. Salton has suggested that content analysis of the titles of cited articles be used to supplement the set of words extracted from the source document itself as index- 78 ing terms. A somewhat related idea was proposed by Garfield who suggested that the source document might be indexed by looking at how the cited documents had been indexed.7 9

Advantages. One of the main advantages of the citation- count, whether for determining citation properties directly or as a measure of some other property (use of the literature, 80 spread of an idea, etc.) is that it is an unobtrusive measure; the act of counting does not change the behavior of the person who is doing what is being observed. It also remains constant with respect to time, therefore is an extremely reliable method (the results of any citation-count can be duplicated at any time); citations made at some time in the past can be counted years later and the properties of citations made at 30

various periods in time can be compared at any later time,

since the citation remains as a permanent record in the docu­

ment. A further advantage is that the citation-count is a

quantitative measure and requires little or no judgment on

the part of the counter.

Some of the advantages of the use of the citation as an

index entry, compared with traditional subject indexing,

include the elimination of problems of changing terminology in

the subject field, static classification schemes, indexer

errors, indexer inconsistency (with self and others), non-

specificity of available index terms, and other inherent

limitations of a priori indexing. No indexer decisions are needed since the necessary relevance judgments have already been made; the association of ideas as indicated by the author

is used rather than an artificial language. While the litera­

ture grows exponentially, the relationship of one document to another changes very little. The number of indexers (the author who provides the citations) will automatically grow with the growth of the literature. Using the citation as

the index entry also avoids the difficult problem of trying to provide subject indexing for all possible future uses, and to decide what will be relevant to any given problem at any given time. The author himself may not always be aware 31 of the implication of his own discoveries of what future 81 uses may be made of his paper. Many subject indexes make

little or no attempt to relate a document to those already

in the collection, while the citation does so implicitly.

In many cases the citation represents a concept which may be difficult to reduce to a linear, linguistic expression.

The citation index will have all the advantages given

for using the citation as an index term to the literature.

In addition, it provides the mephanism for going forward into the literature as well. Since no index terms need to be assigned, the preparation work is primarily clerical in nature. Citation indexes can be produced, and used with little intellectual effort. The index, in effect, becomes self- organizing since each new citation in some way modifies the pre­ vious information; criticisms, corrections, etc. of earlier papers can be readily located and thereby, hopefully, help to minimize future citation of already recognized poor data.

Science Citation Index is multidisciplinary and therefore has the potential to reveal relationships to documents in other fields which would be missed with the use of conventional sub­ ject indexes. In their comparisons of Science Citation Index Op Do QJi with subject indexes, Ghosh, Huang, and Spencer all found that Science Citation Index gave a larger number of relevant citations over the initial time span studied. 32

Disadvantages. Since not a great deal is known about how

citations operate in the communication of science, most of the

disadvantages discussed in the literature are primarily con­

cerned with the assumptions involved in relating the cited

document to the source document (and the source document to the

cited document) and in relating citation-counts to some measure

of some other property of the literature. There is particular

concern that the citation-count must not be equated entirely

with the scientist's use of the literature and especially not

to the whole system of scientific communication.

The first major criticism of the citation-count was that

of Brodman who compared the ranking of journal titles obtained

by citation counts and by expert opinion and, obtaining a low

correlation value, concluded that the method of citation count- op­ ing was invalid. ° Postell compared Brodman1s lists of journals

against circulation records at his University and found a better correlation with the expert opinion list than the 86 citation-count list. Brown suggested that Brodman's lists of journals were actually taken from two different populations and at two different stages in the communication cycle: a more general list (the citation-count) and a specialized list (the expert opinion). He further suggested that lack of correlation 87 did not indicate which was the invalid method. In her review 33 of Brown's book, Brodman not only did not discuss his arguments

but goes on to suggest that Brown found out about her article

only after he completed his book and "for perfectly normal,

psychological reasons could not bring himself to admit that

the years of effort put into compiling his list might be 88 wasted."

The main sources of Brodman's criticisms were the assump­

tions (1) that the value of a journal is in direct proportion •

to the number of times cited, (2) that a key journal can be representative of the field, and (3) that if several key journals are chosen, equal weight can be given to the citations

from each. Another common complaint is that a journal with many shorter articles will be cited more than a journal with fewer longer articles, that is, the number of articles per year per journal might also be an important factor.

A problem related to the selection of the source journals is that the particular time period chosen for the study may not be either representative or'typical. Another time dis­ advantage is the existence of the lag period, the time period between the completion of the research and the publication of the article, a period during which no citations are made. Thus citation-counts rarely include citations a month or two old while other literature counts do indicate the use of this very 34

recent literature. In citation-counts, the date of publica­

tion is used for time distribution studies, rather than the

date of submission of the article, or even more important the

date of the actual use of the citation. Time distribution

studies also incorporate the systematic error introduced by .•

rounding-off the date to the year, rather than the month or

even day of publication.

Other arguments against the citation-count as a measure

of literature use are the problems of underrating, not all

items used were cited, and overrating, not all items cited were

used. Citations only indicate what material was known or

available to the author and not a choice of the best from all

existing material; "citations are ultimately linked up with 89 the degree of physical and bibliographical accessibility." ^

A referee of an article dealing with a citation-count has even

suggested that "clearly one's ability to cite is a function of

one's ability to read—not of the volume of potentially readable 90 material."'7 The citation-count as a measure of literature use does not give any information about the secondary sources

which may have been used. Another limitation of the use of

citations as an indication of the materials which the author has used in the preparation of his article are the editorial 35 demands and limitation placed on the allowable number of 91 citations or items in bibliographies.

Some information is common knowledge and therefore is not cited when used. This can be classical knowledge or it can even be some very recent information which has already come

to be accepted in the invisible college but which does not have a source document which can be cited.

Indiscriminate use of citation-counts also involves the assumption that "the most important technical contributions are 92 based on previously published work" thus ignoring the importance of oral communication. There is also the argument that "a simple citation count may tend to favor applied research paper's at the expense of the fundamental contribution on which 93 they .ace based."^

There is also the criticism that authors tend to cite themselves, to cite the journal in which they publish, and to cite primarily articles in their native language. Authors of scientific papers may not be representative of the whole class of users of scientific information, and total library use is not likely to be identical with the research use which citation studies are designed to measure.

Many of these criticisms of the citation-count must, of course, be examined in light of the purpose of the specific study. 36

Citation indexes also have disadvantages. The previously mentioned lag period is of prime importance to the researcher who desires to keep up-to-date with the literature of his field. Added to this lag period is the further delay in publi­ cation of the citation index, thus diminishing its importance as a current awareness tool. The use of citation indexes also necessitates the knowledge of several articles pertinent to the search problem in order to gain access to the information.

There is also the problem of incorrect bibliographic citation, mistakes by the author both large (which do not lead to the correct document) and small (which eventually lead to the correct document but which are then separated from the correct biblio­ graphic citation in the index). Another problem of citation indexes related to errors and the separation of different forms of the same citation occurs when an article appears in several different sources (or when several preliminary communications appear before the complete research article is published) in essentially the same form. All of these may be cited in various source documents but the citation index will not bring them together.9 4 Cuadra also remarks on this idea when he proposes that one reason for finding only a few common citations is that writers repeat themselves in various articles and therefore the same information is available for citation in several different 37 95 places. If the researcher knows of only on

May has also express?cd the fear that the existence of citation indexes, rather than improving citation habits as

Garfield suggests, will lead to further abuses such as citing friends (those who cite back), longer lists of citations (so as to be the source article more often), and citing obvious sources.9 6

Another danger is in not recognizing the limitations of using the number of citations to an author's work as a measure and equating quantity with quality, or impact with significance.

The use of the citation as a qualitative measure ignores the fact that the source document may be very critical of the cited document and therefore the citation might better have a negative rather than a positive value. The significance of a document may not yet be recognized by contemporaries, and the work of greatest significance might be so well known that it is no longer cited. Citations to previous articles (often a self citation) which express a certain opinion are often used to support that same opinion. Kuppers suggests that the use of citation indexes to measure quality will make scientists extremely reluctant to enter anything but the most popular areas 38

of research (so as to have a better chance of being cited) and

thus may prove more vicious in the long run than measuring quality by the number of publications.9 '7

Noise. One of the major criticisms of the use of the citation-tool is the large number of cited documents retrieved which are determined to be not relevant to the search problem.

This so-called "noise" is the result of the assumption that the relationship between source document and cited document is the same for all the cited documents. As a result of this same assumption of similar relationship or similar relevance between source document and cited document, each citation is given equal weight with every other citation in citation-counts.

Matthews discusses this problem in terms of the "unspecified citation": "an examination of the citations listed in the index will not specify how relevant the citations are to the original paper, to each other, or to the purposes of the reader." He gives the specific example of when the citation "may have been used to document a procedure or a statistical process which 98 has little to do with the purpose of the paper."^ A vivid illustration of this can be found in many of the articles describing citation indexes where the citation is related to the source document in such a unique way that probably only the author will ever use that relationship (it is doubtful 39 that any scientist interested in "Protein Measurement with the

Folin Phenol Reagent" would care to be lead to an article on 99 citation indexing).

Melin related the problem somewhat to the amount of material involved; "in general, a citation implies a relationship between a part or the whole of a cited paper and a part or the whole of the citing paper."

*• In his discussion of Kessler's bibliographic coupling,

Martyn indicates that even if document A and document B do share a reference to document C (the definition of bibliographic coupling), the cited information from document C may not be the same for both A and B and concludes that "bibliographic coupling is not a constant unit and a unit which is not constant is not a unit.•4. iilO" l

Tagliacozzo has remarked upon this problem of the retrieval of irrelevant documents and suggests that "there are many kinds of relationships connecting one paper to another, and that the relationship between a reference and the citing paper may not be the one relevant to a particular search."

One unit of measurement used by Huang in his comparison of citation and subject indexes was the noise ratio (defined as the number of non-pertinent unique items/total unique'items retrieved). The noise ratio for the citation index was con- 40 siderably higher than that for the subject index. He suggested that "the main source of this overwhelming noise is the multi- unit nature of documents, i.e., documents containing more than one unit of information." He also found that "the noise ratio increases with each step of a backward search or forward search. 103 Thus it is essential to eliminate noise at each step."

Spencer also mentions the problem of noise in her study on SCI and recognizes that only a few citations may deal with the aspect of the topic of the source document in which the 104 researcher is interested.

Scientists themselves indicated in a use study conducted by the Great Britain Council on Scientific Policy "that there is a strong desire for a system of grading references according to their usefulness." ^

The usual method of reducing noise is to use the title of the cited document as an indication of the topic covered in the document. However, in many scientific journals the title of the document is not included in the bibliographic citations.

As a result, titles are not included in the citation index of

Science Citation Index.

Lipetz has discussed this problem and proposed the use of

"citation relationship indicators" in citation indexes to help the searcher decide if the source document is likely to be 41 relevant to the search problem, "for the dilemma of the user of the citation index is that he knows from experience that only a fraction of the references which cite his starting reference lo6 will be relevant to his search requirement." In her study of the classification of the various uses made of citations by biochemists (from inspection of the source document), Magee 107 identified six main classes and eighteen subclasses. '

Thus it would appear that the relationship between the source document and the cited document is not constant but rather changes from citation to citation and from search problem to search problem.

Citation Habits. Related to this problem of noise and the relevance of the cited document to the source document is the very important criticism leveled against the basic assumption that the citations reflect earlier work related to the research and only the research which is being described.

Examples of such criticism include May, "the author selects citations to serve his scientific, political, and personal goals 108 and not to describe his intellectual ancestry"; Titley, "it is obvious that authors prepare their bibliographies for many devious reasons and in many devious ways...'upmanship' factors such as honesty, sophistication, tact and necessity are more apt to determine what references are used rather than results 42 109 of a literature search"; * and Kaplan, "the citation practices

of scientists today are in large part a social device for coping

with problems of property rights and priority claims. Only

incidentally do these citations serve as an actual, accurate

reconstruction of the scholarly precursors of one's own

contribution."

One norm of the social system of science which may be operating in the production of citations is that the scientist

exhibit the proper degree of humility regarding his work. One method of displaying such humility is to admit publicly that his work is based on that of others through the use of citations to

this previous work. Citations give respectability and prestige

to scientific writing, and an article lacking footnotes might be considered with suspicion as presenting only personal opinion.

Thus citations may be given in order to conform to the general 111 expectations of scholarly research.

Other factors will cause one document to have a greater possibility of being cited than another equally relevant document.

Merton suggests that "a scientific contribution will have greater visibility in the community of scientists when it is introduced by a scientist of high rank than when it is introduced by one who has not yet made his mark," and supports this idea with evidence from Zucherman's study on American Nobel Prize winners 43 in science. The Nobel laureates discussed the effect of winning on their authorship habits and suggested that when their name now appears on an article, the contributions to the article by co-workers will be completely overlooked, yet at the same time if their name did not appear, the contribution of the article 112 ' might be missed.

Certainly not enough is known about the citation habits of scientists and the related larger problem, the writing habits of scientists.

Merton, one of the first investigators in the area of the sociology of science, suggests that "one of the important institutional norms of science is originality"; thus priority becomes very important in achieving the rewards of science

(eponymy, prizes, medals, memberships in societies, etc.), and

¥ publication is the method by which priorities and property rights 113 are established. ' The idea that the majority of papers are published for the peer group is expressed by Price who feels that the assumption "that each man is entitled to present his findings and where his paper is subject to scrutiny by a jury 114 of colleagues" is basic to the motivation of scientists, and by Kilgour who says, "it appears that it is evaluation by colleagues leading to recognition... that motivates an author to publish a new finding. Achievement of recognition and subse­ quent reward is the principle motivation for publication.'.' * 44

Another primary reason for publication of results is of course the idea of "publish or perish", where the quantity of publication is often used as a measure of ability and "the road 116 to academic heaven is paved with publications." Margolis has also suggested that publication is used to account (to the institution, funding agency, etc.) for the time and money spent on the research. He feels too, that "the more papers are written, the less they count for and the greater is the pressure 117 to publish more."

Thus it would appear that the scientist is not writing primarily to present information or data, or for archival storage, but rather to advance knowledge, establish priorities, and become visible in his peer group. If the scientist writes accounts of his research for various social reasons in addition to the desire to advance knowledge in his field, this will in turn certainly affect his citation habits as well.

Hypothesis

The assumptions that authors tend to have similar citation habits and that some constant relationship exists between source articles and their cited articles are basic to the two major uses made of citations: the citation-tool (the citation as an indication of subject area) and the citation-count (the citation 45 as an indication of literature use). In this study it is conjectured that, while authors do tend to have similar citation habits, citations do in fact reflect differing rela­ tionships to their source articles.

As Kessler noted "the 'scientific paper' is such a marvelous and accommodating invention that we seem to take it for granted and forget that it has form and structure that fits 118 its function" (italics mine). In the field of chemistry, the journal article presenting the results of original research can be divided into three sections, the introduction, the experimental, and the results/discussion. (It is suggested that the formal article, divided into these three recognizable sections, might be the result of the editorial requirements of the specific journal as well as the influence of some other types of writing such as the senior research paper, the thesis, etc., a preliminary source of a large percentage of publication in the field of chemistry.)

In this study it is proposed to take advantage of this form and structure in an investigation of some citation properties.

It is suggested that the level (or amount) of material from the cited article, actually being cited by the source article, will vary significantly with,the section in which the citation occurs because properties of citations vary with the section L k6 in which the citation occurs .and citation level is a property of citations. It is specifically hypothesized that, in the literature of a specific field of an experimental science

(in this case organic chemistry), citations in the introduction section will be to significantly larger amounts of cited material (the whole of the cited article), citations in the results/discussion section will be to significantly smaller amounts of cited material (a part of the cited article), and citations in the experimental section will be to small pieces of cited material (words from the cited article).

In order to test the specific hypothesis, that the level of cited information from the cited article varies with the section of the source article in which the citation occurs, source articles were chosen from the literature of organic chemistry and their citations investigated with respect to the level (or amount) of material actually being cited. The two properties of citations which are most often studied are the journal distribution and the time distribution. Therefore journal title and year of publication data were collected for these citations in order to investigate the journal distribu­ tion and the time distribution with respect to the section in which the citation occurs, as a test of the idea that properties of citations in general vary with the section. 47

The reason proposed for the existence of these differences

in citation properties with respect to section, is that the

three identified sections of the journal article are an indi­

cation of three different needs of the scientist, and different

needs require different types and amounts of information. It

is further suggested that the three sections can be equated to

the three stages of the original experimental research which has been conducted and which is being described in the article,

the background preparation or hypothesis formulation (intro­

duction), the actual experiment or collection of the data

(experimental), and the analysis of the results (results/

discussion).

The type of information needed from the literature will

then vary with the stage of research; the background will require general and specific information on the stage of develop­ ment, the experiment will require specific details concerning synthesis and identification of chemical compounds, techniques, apparatus, etc., and the analysis will require information about the research problem as a whole and the particular steps within it. It is suggested that these information needs will be satisfied in general, by differing amounts of material.

Some support for these ideas can be found in the recent studies conducted on the problems of relevance, where the 48

stage of research was identified as a variable In determining 119 relevance, and the ideas expressed in use studies (and 120 reviews of use studies) that different needs do exist and

are satisfied by the literature (both sources and channels used to find these sources) in different ways. Garvey and

Griffin have also suggested that the needs of the scientist are not constant but that "these needs, which change from time to time, are determined by the subject matter of his research, his own mode of working, his attitudes towards communicating 121 his own work, the stage of his research, and so on." Rath has suggested that the phase of research or "phase effect" will determine the literature demand patterns, and identified three 122 phases, search, experiment, and analysis.

Egan and Henkle have identified eight steps in the research process and indicated the corresponding probable use of recorded knowledge. These again are readily compressed into three major stages; (1), (2) and (3) deal with problem presentation, (4),

(5) and (6) deal with methodology for both data collection and analysis and with the search for the data, (7) and (8) deal with conclusions, implications, consequences.12 3J

Somewhat related to the idea that the stage of research will determine citation properties, is the idea behind "Group

1. Original scientific contribution or intent of citing paper" 49 of Lipetz's "citation relationship indicators" which are intended

to indicate to the searcher some idea of the relationship between

the source document and the cited document. Examples from

Group 1. include hypothesis or theory, statement of experimental 124 technique, explanation. Many of these indicators could readily be related to the individual sections (instead of the whole citing paper). Magee also remarks on the formal research article and used the sections as a basis for her classification of the use of cited material by biochemists. She actually used five sections, separating the results from the discussion and adding a conclusion section. However, as she indicates, the 12*5 results and the conclusion should not have any citations.

It is not suggested that these stages of research are necessarily sequential activities, that is that one stage is completed before the next one is begun. The research may already be in progress when an article is discovered which is then cited as background material. One paper may present the combined results of several separate studies. Medawar has referred to the formal presentation of the journal article and argues that the process and progress of research are not 126 as clear-cut as the journal article might indicate. It is also realized that the journal article is only a summary of the research and therefore the reverse is not claimed, that 50

is, that all needs are represented by the journal article or

that all stages involved in the research are represented by the

article (in fact the writing of the article itself can be

considered a fourth stage).

As well as representing the three stages of research,

the three sections of the source article might also be thought

of as examples of the three major approaches to the literature used by the scientist in his research. Voigt has identified

these three as (1) the current approach (keeping-up-to-date),

(2) the everyday approach (specific information on specific occasion), and (3) the exhaustive approach. ' The experimental

section is readily identified with (2), the introduction section with (3)> while the results/discussion section comes closest to (1). It is expected that the results/discussion section, in any suggested model for the three sections, must in fact have some of the characteristics of both the introduc­ tion and the experimental sections since the resuits/discussion can not avoid discussion of the research problem as a whole or specific experimental details.

In most of the citation-count studies of both journal distribution and time distribution discussed earlier, different values obtained for the constants appearing in the proposed mathematical models (usually the slope of the calculated best 51 fitting straight line) are attributed to differences in subject fields. Stevens has presented evidence to indicate that title scatter of citations is greater for technologies than for pure sciences, and for pure sciences in the early stages of develop- 128 ment than when well established. Related to this, Stevens has also shown that the research method used will determine 129 the title scatter of citations. MacRae and Adams have also suggested that the intent of the citation will produce two different types of citation which need to be differentiated,

"citations furnishing the data under study from those referring 130 to interpretation by other scholars" and "inspection reveals that the references are of two types, those pertaining to data, and those pertaining to concept."13 1 Price has also shown the existence of two types of citations, those to the classical 132 literature and those to the ephemeral literature. Thus there is evidence from previous studies of a non-constant relationship between source document and cited document.

Related to the idea that each citation should not be given equal weight is the fact that the same reference may be used several times, in different sections and for different purposes throughout the article. Most citation counts consider only the number of bibliographic citations. The only known'example where the difference between the citation and the bibliographic 52

citation is taken into account is that of Magee who states "it

became evident immediately that each citation to the reference

must be used as the unit for classification since this citation

was to a distinct section of the reference and the information

cited at each point was used to satisfy a different need." ^

In addition, since the section in which the citation appears

is important in this study, the introduction would receive a

disproportionate share by virtue of appearing first in the

article. Therefore, two categories of citation are maintained

throughout the study, the number of unique citations (the number

of bibliographic citations) and the number of gross citations

(the number of unique citations plus the number of repeat

citations).

It is suggested that different information sources are required to different extents during the course of a research project and that perhaps the organization of information services should take these differences into account. The idea that dif­ ferent levels of information are being used (cited) has impli­ cations for indexing problems, since this might indicate that a document should be indexed at different levels to satisfy these different needs. (The level of indexing is not to be confused with the depth of indexing which has come to mean only the number of indexing terms provided for a document and says 53 little or nothing about the level (or amount) of material from the document which is indexed.)

Another implication for citation indexing is the suggestion that the level of material actually being cited might be some indication of the strength of the relationship between the cited article and the source article (e.g., the larger the amount cited, the closer the relationship). If the section of the source article in which the citation occurs is directly related to the level, then an indication of section would infer both the closeness of the relationship as well as the relationship to the stage of research.

It was hoped that the tracings for the available indexing for the cited articles and the source articles could be obtained from Chemical Abstracts so that some comparison could be made of the level of indexing provided and the level of indexing needed

(as indicated by the citations in the source article). However, this information was not readily accessible at present. 54

Footnotes

Norman Kaplan. Norms of Citation Behavior; Prolegomena to the Footnote. American Documentation, 16, 179-184 (I965).

Rolland E. Stevens. Characteristics of Subject Litera­ tures. Chicago, ACRL, 1953. (ACRL Monograph No. 6).

-Trank S. Sullivan. A Garland of Ibids for Van Wyck Brooks. New Yorker, 17_, 15 (April 19, 1941).

Philip H. Rinelander. Notes on Footnotes. College and Research Libraries, 2J, 184 (1966).

"'American Chemical Society. Handbook for Authors of Papers in the Journals of the American Chemical Society. Washington, D. C, American Chemical Society Publications, I967.

S. Dutta. Literature Citations in Scientific and Technical Periodicals--A Survey. Journal of Scientific and Industrial Research (India), I7A, 259-261 (1958).

' E.g,,, * Paul Dunkin. Happiness is a Long Footnote. Library Resources and Technical Services, J_, 403-405 (I963); Phyllis A. Richmond. Misery is a Short Footnote. Library Resources and Technical Services, 9_, 221-224 (I965); Mary R. Kinney. The Abbreviated Citation—A Bibliographical Problem. Chicago, ACRL, I967. (ACRL Monograph No. 28). o Wyandotte-ASTM Punched Card Project. CODEN for Periodical Titles. Philadelphia, American Society for Testing and Materials, 1966. q ^American Standard for Periodical Title Abbreviations. Columbus, Ohio, National Clearinghouse for Periodical Title Word Abbreviations, I963. Larry J. Barr. Origin of Footnotes. , RQ, £, I6-I7 (I966). 11 R. E. Kirkwood. Footnotes. RQ, £, 29 (1966); Mrs. William S. Forshaw. . . .And More Footnotes. RQ, £, 29 (I966). 12 Cyril C. Barnard. Bibliographic Citations. Librarian and Book World, 3£, 105 (1950). 55 13 ~\J. G. Hodgson. The Preparation of Bibliographies and Bibliographical Footnotes and Citations. Fort Collins, Colorado, Library, Colorado A & M College, 1956. p. 6. 14 Derek de Solla Price. Little Science, Big Science. New York, Columbia University Press, I963. pp. 64-65. 15 "^Norman Kaplan. Norms of Citation Behavior: Prolegomena to the Footnote. American Documentation, 16, 180 (I965).

Paul S. Lykoudis, P. E. Liley, and Y. S. Touloukian. Analytical Study of a Method for Literature Search in Abstracting Journals. In International Conference on Scientific Information, Washington, 1958. Proceedings, pp. 351-375; A. 0. Cezairliyan, P. S. Lykoudis, and Y. S. Touloukian. A New Method for the Search of Scientific Literature Through Abstracting Journals. Journal of Chemical Documentation, 2_, 86-92 (I962).

17'A . Stephanie Barber. A Critical Review of the Surveys of Scientists' Use of Libraries. In The Provision and Use of Library and Documentation Services, ed. by W. L. Saunders. Oxford, Pergamon Press, I966. p. I52. 18 John Martyn. Examination of Citation Indexes. Aslib Proceedings, l£, 189 (1965). 19 E. M. Keen. Citation Indexes. Aslib Proceedings, 16, 246 (1964). ~~ 20 P. L. K. Gross and E. M. Gross. College Libraries and Chemical Education. Science, 66, 385-389 (1927). 21 Charles Harvey Brown. Scientific Serials. Chicago, ACRL, 1956. (ACRL Monograph No. 16). pp. 101-102. 22 Charles Harvey Brown. Scientific Serials. Chicago, ACRL, 1956. (ACRL Monograph No. 16). 23 JS. C. Bradford. Sources of Information on Specific Subjects. Engineering, 137« 85-86 (1934); Documentation. London, Crosby, Lockwood, 1948. pp. 110-120. 24 Brian C. Vickery. Bradford's Law of Scattering. Journal of Documentation 4, 198-203 (1948). 56 25 0. V. Groos. Bradford's Law and the Keenan-Atherton Data. American Documentation, 19_, 46 (1967). 26 B. C. Brookes. The Derivation and Application of the Bradford-Zipf Distribution. Journal of Documentation, 24, 252 (1968). 27 'Dale Lockard Barker. Characteristics of the Scientific Literature Cited by Chemists of the Soviet Union. Unpublished Ph. D. dissertation, Graduate School of Library Science, University of Illinois, 1965. pp. 97-99. 28 M. G. Kendall. The Bibliography of Operational Research. Operational Research Quarterly, 11, 31-36 (i960). 29 ^Alfred J. Lotka, The Frequency Distribution of Scientific Productivity. Washington Academy of Sciences, Journal, 16, 317-323 (1926).

3 Geogge K. Zipf. Human Behavior and the Principle of Least Effort. Cambridge, Mass., Addison-Wesley, 1949. Chapts. 9-10. 31 Dale Lockard Barker. Characteristics of the Scientific Literature Cited by Chemists of the Soviet Union. Unpublished Ph. D. dissertation, Graduate School of Library Science, University of Illinois, I965. PP. 99-103. 32 Charles F. Dalziel. Evaluation of Periodicals for Electrical Engineers. Library Quarterly, £, 354-372 (1937). 33 J*T. F. Cole. A New Look at Reference Scattering. Journal of Documentation, 18, 58-64 (1962). 34 J F. F. Leimkuhler. The Bradford Distribution. Journal of Documentation, 23_, 197-207 (I967). 3^Dale Lockhard Barker. Characteristics of the Scientific Literature Cited by Chemists of the Soviet Union. Unpublished Ph. D. dissertation, Graduate School of Library Science, University of Illinois, 1965. p. 93. 36 M. G. Kendall. The Bibliography of Operational Research. Operational Research Quarterly, 11, 35-36 (i960). 57 37 'B. C. Brookes. The Derivation and Application of the Bradford-Zipf Distribution. Journal of Documentation, 24, 247-265 (1968).

3^I. K. Buckland and A. Hindle. Library Zipf. Journal of Documentation, 2J5,, 52-57 (1969); R. Fairthorne. Library Zipf. Journal of Documentation, 2£, 152-154 (I969). 39 Charles F. Gosnell. Obsolescence of Books in College Libraries. College and Research Libraries, £, 115-125 (1944). 40 P. F. Cole. Journal Usage Versus Age of Journal. Journal of Documentation, 1£, 1-11 (I963). 41 Paul Weiss. Knowledge: A Growth Process. Science, 131, I7I6-I7I9 (I960). 42 Derek J. de Solla Price. Networks of Scientific Papers. Science, 149, 510-515 (I965). ^3R. E. Burton and R. W. Kebler. The "Half-life" of Some Scientific and Technical Literatures. American Documentation, 11, 18-22 (I960). 44 A. J. Meadows. Citation Characteristics of Astronomical Research Literature. Journal of Documentation, 23, 30 (1967). 45 ^Dale Lockard Barker. Characteristics of the Scientific Literature Cited by Chemists of the Soviet Union. Unpublished Ph. D. dissertation, Graduate School of Library Science, University of Illinois, I966. pp. 66-87. 46 H. C. Lehman. Age and Achievement. Princeton, N. J., Princeton University Press, 1953. 47 W. Dennis. The Age Decrement in Outstanding Scientific Contributions: Fact or Artifact? American Psychologist, 13, 457-460 (1958). 48 Duncan MacRae. Growth and Decay Curves in Scientific Citations. American Sociological Review, 3_4, 63I-635 (I969). 58 49 -'Charles P. Bourne. Some User Requirements Stated Quantitatively in Terms of the 90 Percent Library. In Electronic Information Handling, ed. by Allen Kent and Orrin E. Taulbee. Washington, D. C, Spartan, I965. pp. 93-110.

"* M. K. Buckland and I. Woodburn. Some Implications for Library Management of Scattering and Obsolescence. Lancaster, Eng., Library, University of Lancaster, I968. (University of Lancaster Library, Occasional Papers No. 1).

"* John Martyn and Alan Gilchrist. An Evaluation of British Scientific Journals. London, Aslib, I968. (Aslib Occasional Publication No. 1). 52 ^ J. S. Ghosh. Literature Searching with the Science Citation Index: An Experimental Study. Annals of Library Science and Documentation, 14, 133-142 (I967). -'"'Theodore S. Huang. Efficacy of Citation Indexing in Reference Retrieval. Library Resources and Technical Services, 12, 415-434 (1968).

•* Carol C. Spencer. Subject Searching with Science Citation Index. American Documentation, 18, 87-96 (I967).

•'^John W. Tukey. A Citation Index for Statistics and Probability. Institut International de Statistique, Bulletin, 40, 747-756 (1964). 56 J F. L. Goodman. A Citation Index for Literature on New Educational Media. American Documentation Institute. Pro­ ceedings of the Annual Meeting, 33-34 (1963). "''Phyllis Reisner. Pre-test and Potential of a Machine- stored Patent Index. In Kochen, Manfred. Some Problems in Information Science. N.Y., Scarecrow, I965. PP. 161-173; Eugene Garfield. Patent Citation Indexing and the Notions of Novelty, Similarity, and Relevance. Journal of Chemical Documentation, 6, 63-65 (I966). 58 •'Ben-Ami Lipetz. Evaluation of the Impact of a Citation Index in Physics. In International Federation for Documenta­ tion. Conference, I965. Proceedings. Washington, D. C, Spartan, I966. pp. I07-II5. 59 59 •^Sanborn C. Brown. A Bibliographic Search by Computer. Physics Today, 19_, 59-64 (1966). John Martyn. Examination of Citation Indexes. Aslib Proceedings, 1J_, 184-196 (1965).

Andre Deweze. Automated Establishment and Utilization of Bibliographical Citation Indexes. UNESCO Bulletin for Libraries, 18, 211-223 (1964). 62 Eugene Garfield and I. Sher. ISI's Experiences with ASCA—A Selective Dissemination System. Journal of Chemical Documentation, £, 147-153 (I967). 63 3M. M. Kessler. Bibliographic Coupling Between Scientific Papers. American Documentation, 14, 10-25 (I963); Bibliographic Coupling Extended in Time: Ten Case Histories. Information Storage and Retrieval, 1, I69-I87 (I963); Comparison of the Results of Bibliographic Coupling and Analytic Subject Indexing. American Documentation, 16, 223-233 (I965). 64 Oden E. Sheppard. The Chemistry Student Still Needs a Reading Knowledge of German. Journal of Chemical Education, 12, 472-473 (1935). "'A. M. McAnally. Characteristics of Materials Used in Research in United States History. Unpublished Ph. D. disser­ tation, Graduate Library School, University of Chicago, 1951. 66 Herman H. Fussier. Characteristics of the Research Literature Used by Chemists and Physicists in the United States. Library Quarterly, 19_, 19-35, 119-143 (1949). 67 'Robert N. Broadus. A Citation Study for Sociology. American Sociologist, 2_, 19-20 (I967). 68 R. L. Dahling. Shannon's Information Theory. In Studies of Innovation and of Communication to the Public. Palo Alto, Stanford University, Institute for Communication Research, I962. 69 ^Eugene Garfield, I. H. Sher, and R. J. Torpie. The Use of Citation Data in Writing the History of Science. Philadelphia. Institute for Scientific Information, 1964. 60 70 ' Louis V. Xhignesse and Charles E. Osgood. Bibliographic Citation Characteristics of the Psychological Journal Network in 1950 and in i960. American Psychologist, 22, 778-791 (I967). 71 ' Ralph Garner. A Computer-orientated Graph Theoretic Analysis of Citation Index Structures. In Three Drexel Informa­ tion Science Research Studies. Philadelphia, Drexel Press, 1967. 72 ' E. Yagi. Application of a Type of Matrix to Analyze Citations of Scientific Papers. American Documentation, 16, 10-19 (1965). "" 73 ,JJ. H. Westbrook. Identifying Significant Research. Science, 132, 1229-1234 (i960). 74 ' I. H. Sher and Eugene Garfield. New Tools for Improving and Evaluating the Effectiveness of Research. Conference on Research Program Effectiveness, I965. Proceedings. Washington, D. C, I965. PF*. 135-146. 7*5 '•^Stephen Cole and Jonathan R. Cole. Scientific Output and Recognition: A Study in the Operation of the Reward System in Science. American Sociological Review, 32, 377-390 (I967). 76 Alan E. Bayer and John Folger. Some Correlates of a Citation Measure of Productivity in Science. Sociology of Education, 3£, 381-390 (I966). 77 • 'Eugene J. Webb and Jerry R. Salancik. Notes on the Sociology of Knowledge. Journalism Quarterly, 42, 56I-565 (1965). ~~ 78 Gerald Salton. Associative Document Retrieval Techniques Using Bibliographic Information. Association for Computing Machines, Journal, 10, 440-457 (1963)* •^Eugene Garfield. "Science Citation Index"--A New Dimension in Indexing. Science, 144, 649-654 (1964). 80 Edwin B. Parker, William J. Paisley, and Roger Garrett. Bibliographic Citations as Unobtrusive Measures of Scientific Communication. Palo Alto, Stanford University, Institute for Communication Research, I967. 61 81 Eugene Garfield. Citation Indexes for Science. Science, 123, 62 (1956). J. S. Ghosh. Literature Searching with the Science Citation Index: An Experimental Study. Annals of Library Science and Documentation, 14, 133-142 (I967). go ^Theodore S. Huang. Efficacy of Citation Indexing in Reference Retrieval. Library Resources and Technical Services, 12, 415-434 (1968). 84 Carol C. Spencer. Subject Searching with SCI. American Documentation, 18, 87-96 (I967). 'Estelle Brodman. Choosing Physiology Journals. Medical Library Association, Bulletin, 32,, 479-483 (1944). 86 William D. Postell. Further Comments on the Mathematical Analysis of Evaluation of Scientific Journals. Medical Library Association, Bulletin, 34., 107-109 (1946). 'Charles Harvey Brown. Scientific Serials. Chicago, ACRL, 1956. (ACRL Monograph No. 16). pp. 2-11. 88 Estelle Brodman. Review of "Scientific Serials." Medical Library Association, Bulletin, 4£, 114-115 (1957). 89 Th. Friis. The Use of Citation Analysis as a Research Technique and Its Implications for Libraries. South African Libraries, 23_, 13 (1955). 90 'As Quoted By: M. C. Gomparts. Law of Constant Citation: Mr. Gomparts Replies. Journal of Documentation, 25, 62-63 (I969). 91 ' Dell Lebo. Publication Versus Communication. Science, ]£8> ^-^5 (1958). ' " Charles F. Dalziel. Evaluation of Periodicals for Electrical Engineers. Library Quarterly, J_, 357 (1937). 93 J. Margolis. Citation Indexing and Evaluation of Scientific Papers. Science, 155, 1217 (I967). 94 'Seymour Lubetzky and R. M. Hayes, Bibliographic Dimensions in Information Control. American Documentation, 20, 247-252 (1969). 62 "'Carlos A. Cuadra. Identifying Key Contributions to Information Science. America^ Documentation, 15, 289<-295 (1964). 96 Kenneth 0. May. Abuses of Citation Indexing. Science, 156, 890, 892 (1967). 97 "J. R. Kuppers. Literature Citation Counting. Science, 133, 1138 (1961). 98 ' G. M. Matthews and J. Van Luik. Citation and Subject Indexing in Science. Library Resources and Technical Services, 9_, 478-482 (1965). 99 Eugene Garfield. Citation Indexes in Sociological and Historical Research. American Documentation, 14, 289-291 (I963). Morton V. Malin. The Science Citation Index: A New Concept in Indexing. Library Trends, 16, 376 (I968). John Martyn. Bibliographic Coupling. Journal of Docu­ mentation, 20, 236 (196,4). 102 R. Tagliacozzo. Citations and Citation Indexes: A Review. Methods of Information in Medicine, 6, I36 (I967). 103 Theodore S. Huang. Efficacy of Citation Indexing in Reference Retrieval. Library Resources and Technical Services, 12, 442, 432 (1968). 104 Carol C. Spencer. Subject Searching with SCI. American Documentation, 18, 87-96 (I967). 105 Great Britain Advisory Council on Scientific Policy. Survey of Information Needs of Physicists and Chemists. Journal of Documentation, 2_1, 87 (I965). Ben-Ami Lipetz. Improvement of the Selectivity of Cita­ tion Indexes to Science Literature through the Inclusion of Citation Relationship Indicators. American Documentation, 16, 81-90 (1965). 107 'Marguerite Magee. How Research Biochemists Use Informa­ tion; An Analysis of Use of Information from Cited References. Unpublished M. A. dissertation, Graduate Library School, University of Chicago, I966. )

63 lo8 Kenneth 0. May. Abuses of Citation Indexing. Science, 156, 892 (1967). °*Joan Titley. Citation Upmanship. Rub-off, 16, 4-6 (1965). "~ Norman Kaplan. Norms of Citation Behavior; Prolegomena to the Footnote. American Documentation, 16, 181 (I965).' Ill * —- R. Tagliacozzo. Citation and Citation Indexes: A Review. Methods* of* Information in Medicine, 6, 136 (I967). 112 Robert K. Merton. The Matthew Effect in Science. Science, 159, 56-63 (1968). 113 Robert K. Merton. Priorities in Scientific Discovery: A Chapter in the Sociology of Science. In The Sociology of Science, ed. by Bernard Barber and Walter Hirsch. New York, Free Press of Glencoe, I962. pp. 447-485. 114 Derek J. de Solla Price. Ethics of Scientific Publication. Science, 144, 655 (1964). 115 'Frederick G. Kilgour. Publication of Scientific Discovery: A Paradox. American Documentation Institute. Proceedings of the Annual Meeting, 427-430 (I966). ' 116 F. Reif. The Competitive World of the Pure Scientist. Science, 134., 1959 (1961). 117 J. Margolis. Citation Indexing and Evaluation of Scientific Papers. Science, 155, 1213-1219 (1967)• 118 M. M. Kessler. Some Very General Design Considerations. In TIP System Report. Cambridge, Mass., Massachusetts Insti­ tute of Technology, I967. As Quoted By Joseph H. Kuney. Publication and Distribution of Information. In Annual Review of Information Science and Technology, v. 3. Chicago, Encyclopedia Britannica, I968. p. 33. 119 'Alan M. Rees and Douglas G. Schulz. A Field Experi­ mental Approach to the Study of Relevance Assessments in Relation to Document Searching. Cleveland, Ohio, Case Western Reserve University, Center for Documentation and Communication Research, I967. 2 v. 64 120 E.g. William J. Paisley. The Flow of (Behavioral) Science Information: A Review of the Research Literature. Palo Alto, Stanford University, Institute for Communication Research, I965. 121 W. D. Garvey and B. C. Griffith. Communication in a Science: The System and its Modification. In Symposium on Communication in Science: Documentation and Automation, London, I966. Communication in Science, ed. A. de Reuck and J. Knight. London, Churchill, I967. p. 23. 122 G. J. Rath. Research on the Management of Research and Development. In Second Congress on the Information System Sciences, 1964, ed. by J. Spiegel and D.E. Walker. Washington, D. C, Spartan, I965. pp. 435-444. "^Margaret Egan and Herman H. Henkle. Ways and Means in which Research Workers, Executives, and Others Use Information. In Documentation in Action, ed. by Jesse H. Shera and others. New York, Reinhold, I956. pp. 137-159. 124 Ben-Ami Lipetz. Improvement of the Selectivity of Citation Indexes to Science Literature through the Inclusion of Citation Relationship Indicators. American Documentation, 16, 81-90 (1965). 125 'Marguerite Magee. How Research Biochemists Use Informa­ tion: An Analysis of Use of Information from Cited References. Unpublished M. A. dissertation, Graduate Library School, University of Chicago, 1966. 126 P. B. Medawar. Is the Scientific Paper a Fraud? Tappi, 4J_, 30A, 34A, 40A (1964). 127 'Melvin J. Voigt. The Researcher and His Sources of Scientific Information. Libri, 9_, 177-193 (1959); Scientists' Approach to Information. Chicago, ACRL, I96I. (ACRL Monograph No. 24). 12 8 Rolland E. Stevens. Characteristics of Subject Literatures. Chicago, ACRL, 1953. (ACRL Monograph No. 6). 129 'Rolland E. Stevens. The Use of Library Materials in Library Research: A Study of the Effect of Differences in Research Method. Unpublished Ph. D. dissertation, Library School, University of Illinois, 1951. 65 13J 0Dunca n MacRae. Growth and Decay Curves in Scientific Citations. American Sociological Review, 3,4, 634 (I969). 3 w. Mansfield Adams. Relationship of Keywords in Titles to Reference Cited. American Documentation, 18, 30 (1967). ^Derck J. de Solla Price. Networks of Scientific Papers. Science, 149, 510-515 (I965). 33Marguerite Magee. How Research Biochemists Use Information; An Analysis of Use of Information from Cited References. Unpublished M. A. dissertation, Graduate Library School, University of Chicago, I966. p. 10. 66

PROCEDURE

Selection of Citations

The choice of source documents or journals in citation- count studies can occasion criticism, and to date no fully satisfactory selection method for selecting typical journals has been proposed. The usual procedure by which these source journals are chosen is to formulate certain criteria appropriate to the purpose of the particular study and then to select journals fitting these criteria. The criteria used in this study were that the journal should (1) contain full length articles presenting reports of completed original research

(including experimental details), and (2) deal solely with the field of organic chemistry, and (3) be in the English language.

The Journal of Organic Chemistry was chosen as one source journal which readily meets these criteria. It was also decided to use the organic section of the Journal of the American

Chemical Society since, in all citation-count studies in the field of chemistry, it is regularly identified as the most frequently cited journal. These are the two major journals published in the United States of America in the field of 67

organic chemistry, and since the number of source articles to

be studied was necessarily limited, it was decided to confine

the investigation to these two journals. Another reason for

using these two journals is that they are both published by

the American Chemical Society and therefore reflect the same

directions for authors of papers submitted for publication.

Volume 88 (1966) of the Journal of the American Chemical

Society (hereafter referred to as JACS) and Volume 31 (19^6)

of the Journal of Organic Chemistry (hereafter referred to as

JOC) were arbitrarily chosen for the source years.

These two source journals were also selected because of

the format of the articles appearing in them. The organic

chemistry journal article, as it appears in JACS and JOC,

consists of three quite distinct sections. The article usually begins with some background material relating the

research to previously published work in the same area. This

introduction section is not usually labeled as such. Also,

in a few cases, citations are made from the title of the article (of necessity to articles by the same author and often

in the same series as the source article). Since these citations

from the title were considered to be of a background nature,

they were counted as being in the introduction section. The experimental section, giving primarily the results of the 68

preparation, separation, and identification of the chemical

compounds involved in the research (whether or not the prepara­

tion of these chemical compounds was the sole purpose of the

research) appears at the end of the article, clearly labeled and in smaller type than that of the rest of the article. The

remainder of the article consists of the presentation and dis­ cussion of the results of the research. In some cases there are two separate sections, one labeled "results", the other labeled "discussion"; however, they are often considered

together in one section as "results and discussion" and so they could not be separated in this study. When no results/ discussion heading was used, the introduction was considered to end when any mention was made of what had been done (as opposed to what was going to be done) or of any results obtained

•\ in the research. In all cases, this division between intro­ duction and results/discussion was readily recognized. Thus the three sections (in the usual order of appearance) are the title/introduction, the results/discussion, and the experimental.

These two journals also follow the same format with respect to citations; the reference is given in a footnote at the bottom of the page, with the citation being indicated by a numerical superscript in the text. The bibliographic citation includes authors' names, journal title, page, volume, and year. 69

The title of the cited article is not given. The page listed

is always the first page of the cited article (which therefore

may not be the exact page where the cited information can be

found). If a second citation is made to that footnote some­

time later in the article, a new footnote is not made; instead

the superscript is repeated.

This format leads to an important consideration, one

that is usually ignored in citation counts, that is, the fact

that a citation is frequently used more than once, or repeated.

This consideration is of special importance in this study

since the section in which the citation occurs is the inde­

pendent variable. If these repeat citations (citations which

refer back to previously used footnotes, with a superscript

out of numerical order) are not considered, then a bias would

be introduced since one section of necessity must precede

another and this relative position would then decide in which

section the citation is to be counted. Therefore, throughout

this study two counts have been kept, the number of unique

citations and the number of gross citations (unique citations

plus repeat citations).

A list of the page numbers for the start of each article was compiled from the table of contents for each issue of the

source journals for the year I966. These page numbers were 70 listed sequentially in order of appearance, and a table of g random numbers was then used to select twenty-five articles from each journal to be studied. These source articles are listed in Table 1 (JACS) and Table 2 (JOC) in the order of their selection.

A data sheet was then drawn up for each citation found in the source articles, giving the number for the source article (the page plus the selection number from Tables 1 and 2) as well as the number (the superscript) given the citation within the source article. Only citations appearing in the text of the article were studied; citations appearing in tables (alphabetic rather than numeric superscripts) were considered to be of a different type and so were not investi­ gated at this time.

In many cases, more than one bibliographic citation was found in a numbered footnote. Sometimes these are labeled

(a), (b), (c), etc.; other times they are unlabeled and listed together with no distinguishing notation. The first biblio­ graphic citation listed was considered to be the numbered ' citation, the rest were termed either extra labeled citations or extra unlabeled citations where both the numbered citation and the extra citation are categories of unique citations.

Since it was desirabcLe that each unique citation have a 71 Table 1

Source Articles from the Journal of the American Chemical Society, Volume 88, 1966

S. J. Di Mari, J. H. Supple, and H. Rapoport. Mass spectra of naphthoquinones. Vitamin Ki/«0\» (1226)

S. W. Weidman and E. T. Kaiser. The mechanism of the periodate oxidation of aromatic systems. III. A kinetic study of the periodate oxidation of catechol. (5820)

W. Reusch, C. K. Johnson, and J. A. Manner. Radical and thermal rearrangement of a,£-epoxy ketones. (2803)

A. C. Cope, J. K. Hecht, H. W. Johnson, Jr., H. Keller, and H. J. S. Winkler. Molecular asymmetry of olefins. V. Resolution of cis,trans-1,5-cyclooctadiene. (761)

H.-I. Joschek and L. I. Grossweiner. Optical generation of hydrated electrons from aromatic compounds. II. (3261)

E. T. Strom, C. A. Russell, and J. H. Schoeb. Semidones. II. Radical anions of 2,2'-furil, 2,2'-thenil, and 3,3'- thenil. (2004).

M. M. Kreevoy, P. J. Steinwand, and W. V. Kayser. The acid cleavage of allymercuric iodide. (124).

W. M. Jones and D. L. Muck. The mechanism of the alkoxide-induced conversion of N-nitroso-N-alkylamine derivatives to diazoalkanes. (3798)

R. Hoffmann and R. A. Olofson. The dependence of con­ formational and isomer stability on the number of elec­ trons in extended ir systems. (943)

H. C. Brown, P. M. Weissman, and N. M. Yoon. Selective reductions. IX.. Reaction of lithium aluminum hydride with selected organic compounds containing representative functional groups. (1458) 72 11. N. L. Allinger, H. M. Blatter, L. A. ,Freiberg, and F. M. Karkowski. Conformational analysis. LI. The conforma­ tion of cyclohexanone rings in simple molecules. (2999)

12. M. S. Newman and C. Courduvelis. Reactions proceeding by the [3.2.1] bicyclic path. (781)

13. E. L. Eliel, S. H. Schroeter, T. J. Brett, F. J. Biros, and J.-C. Richer. Conformational analysis. XI. Con- figurational equilibria and chromic acid oxidation rates of alkylcyclohexanols. Deformation effects. (3327)

14. K. B. Wiberg and G. M. Lampman. Conformational equilibra­ tion among 1,3-dihalocyclobutanes. (4429)

15. G. R. Jurch, Jr. and T. C. Traylor. Radical-ionic reaction mechanisms. Homolysis of the carbonium ion J:-butyl tropenium peracetate. (5228)

16. S. G. Smith and G. Su. Kinetics of the addition of Grignard reagents to ketones. The addition of methyl- magnesium bromide to 2,4-dimethyl-4'' -methylmercapto- benzophenone. (3995)

17. E. L. Eliel and F. J. Biros. Conformational analysis. XII. Acetylation rates of substituted cyclohexanols. The kinetic method of conformational analysis. (3334)

18. R. A. Benkeser and T. E. Johnston. Factors governing the reaction of the benzyl Grignard reagent. I. Reactions with acetaldehyde, chloral, and trifluoroacetaldehyde. (2220)

19. R. W. Murray and M. L. Kaplan. Sigmatropic reactions in the l,4-bis(cycloheptrienyl)benzene isomers. (3527)

20. G. A. Russell and M. C. Young. Semidones. III. Radical ions derived from ninhydrin and alloxan. (2007)

21. M. E. H. Howden, A. Maercker, J. Burdon, and J. D. Roberts. Small ring compounds. XLV. Influence of vinyl and phenyl substituents on the interconversion of allylcarbinyl-type Grignard reagents. (1732)

22. H.-I. Joschek and S. I. Miller. Photooxidation of phenol, cresols, and dihydroxybenzenes. (3273) 73 23. P. Brown and C. Djerassi. Mass spectrometry in structural and stereochemical problems. CVI. Occurrence of alkyl and aryl rearrangements in the fragmentation of some organic carbonates. (2469)

24. L. Altschuler and E. Berliner. Rates of bromination of polynuclear aromatic hydrocarbons. (5837)

25. R. M. Fantazier and J. A. Kampmeier. Vinyl radicals. III. The thermal decomposition of t-butyl cis- and trans-a,B- dimethylpercinnamates. (5219J "" 74

Table 2

Source Articles from the Journal of Organic Chemistry, Volume 30, 1966

1. S. Boatman and C. R. Hauser. Condensations at the terminal methyl group of anilino derivatives of benzoylacetone and acetylacetone through their dialkali salts in liquid ammonia. (I785)

2. M. Ikehara and I. Tazawa. Studies of nucleosides and nucleotides. XXIX. Direct synthesis of nucleoside-2',3' cyclic phosphates. (819)

3. K. L. Williamson, L. R. Sloan, T. Howell, and T. A. Spencer. Additivity of chemical shifts in the decalin ring system. Determination of configuration of cis-9- hydroxy-10-methyldecalin-2,5-dione. (436)

4. H. S. Aaron, C. P. Rader, and G. E. Wicks, Jr. Azabicyclic alcohols. IV. Stereochemistry of the 1- and 2-hydroxy- indolizindine and hydroxypyrrolizidine systems. (3502)

5. A. W. Frank and C. F. Baranauckas. Trialkyl phosphates. I. Halogenation of trialkyl phosphites in the presence of alcohols. (872)

6. J. P. Freeman. Conjugate addition of the Wittig reagent. (538)

7. R. E. Ireland and R. C. Kierstead. Experiments directed toward the total synthesis of terpenes. X. A stereo­ selective scheme for diterpenoid resin acid synthesis. (2543)

8. A. E. Jacobson. An unusual Stevens rearrangement of a tetrahydropyridinium salt. (I569)

9. W. Reeve and E. Kiehlmann. The reaction of chloral with methyl ketones. (2164)

10. H. E. Smith, J. C. D. Brand, E. H. Massey, and L. J. Durham. Terpenes. III. The nuclear magnetic resonance spectra and absolute configurations of the thujylamines. (690) 75 11. C. A. Brown and H. C. Brown. Catalytic hydrogena- tion. II. A new, convenient technique for laboratory hydrogenations. A simple, automatic device for atmospheric pressure hydrogenations. (3989) 12. R. A. Bell, R. E. Ireland, and L. N. Mander. Experiments directed toward the total synthesis of terpenes. IX. The total synthesis of (*)-hibaene and the oxygenation of some tetracyclic diter- penes. (2536)

13. B. H. Klanderman. Aldehyde synthesis. A study of the preparation of 9,10-anthracenedicarboxaldehyde and other aromatic aldehydes. (26I8).

14. R. E. Moore and P. J. Scheuer. Nuclear magnetic resonance spectra of substituted naphthoquinones. Influence of substituents on tautomerism, anisotropy, and stereochemistry in the naphthazarin system. (3272) 15. S. Murai and S. Tsutsumi. Copper salts catalyzed addition of trichloro- and dichloroacetonitriles to olefins. (3000) 16. H. E. Ungnade and L. W. Kissinger. Esters and ethers of 2-substituted 2,2-dinitro-l-alkanols. (369) 17. D. M. Piatak and E. Caspi. Heterocyclic steroids. VIII. Steroidal oxazines and 2-aza-A-nor steroids. (3935) 18. E. H. White and H. Worther. Analogs of firefly luciferin. III. (1484) 19. A. Takamizawa, Y. Hamashima, Y. Sato, S. Tanaka, H. Ito, and Y. Mori. Studies on pyrimidine derivatives and related compounds. XLI. Reaction of diethyl benzoylphosphonate with thiamine (Takamizawa reaction 3). (2951) 20. R. C. Lamb, L. P. Spadafino, R. G. Webb, E. B. Smith, W. E. McNew, and J. G. Pacific!. The synthesis and decomposition kinetics of some para-substituted trans-7-benzylidenebutyryl peroxides. The excess a,J3-bis(biphenylene)- 2-phenylallyl method for" the determination of initiator kinetics. (147)

21. L. D. Quin and H. G. Anderson. Some new reactions of functions containing secondary phosphorus. (12 06)

22. D. V. Nightingale and D. N. Heintz. Reactions of 14-hydroxy-14-azadispiro[5.1.5.2]pentadec- 9-ene-7,l5-dione and related compounds. I. The 3,11-dimethyl derivative. (361)

23. H. Kwart and E. R. Evans. The vapor phase re­ arrangement of 'thioncarbonates and thion- carbamates. (410)

24. D. Theodoropoulos and I. Souchleris. Toward the synthesis of a heme octapeptide occurring cytochrome c. (4009)

25. T. H. Siddall, III. Further studies of proton non-equivalence in N-substituted amides and related compounds (3719) 77 unique data number, upper case letters were used to differen­ tiate these extra citations listed under a common number.

Finally, a number was added to indicate how many times the citation was repeated in the source article. Roman numerals were used to indicate each repeated use, i.e., the first time the citation was used in regular sequence in the text, no number was added, the second use became repeat I, the third use repeat II, and so on. An example of one such number is

JACS-2004-6-4(a)-I, indicating the source article from JACS beginning on page 2004 chosen sixth, citation number 4(a) the first time it is repeated.

If a repeat citation referred to a numbered footnote with extra citations (either labeled or unlabeled) it was assumed that the repeat citation referred to all the bibliographic citations listed under that number. Another complication was due to the fact that, in several cases, the footnote in some way referred back to a previous footnote. This would happen, for example, in the case of a citation to a book where a specific page was indicated in the first footnote (rather than merely the first page as is done with journal articles). The second footnote would then give only the number of the first footnote plus the new page number. In other cases, the reference would actually be some explanatory comment (non-

i 78

bibliographic) which included a citation to another footnote.

For the purposes of a preliminary count, these refer citations

were considered unique citations, and a separate data sheet

was prepared in the usual manner with the addition of a "see"

note. A corresponding "see also" note was then added to the

data sheet of the referred-to citation.

The authors' names, the journal title, page, and volume

of the cited article were then added to the data sheet,

together with the section of the source article in which the

citation occurred; the title/introduction, the results/

discussion, and the experimental. These sections will be

referred to as T/l, R/D, and Exp respectively in this study.

The data numbers for the non-bibliographic citations as

well as for books, theses, and patents were listed on separate

sheets.

All of this information was obtained from the source

article. The data for the combined total of the number of citations obtained from the fifty source articles are given in Table 3, along with the separate totals for the two source journals, JACS and JOC. As can be seen in Table 3, JACS has a higher percent of repeat citations than does JOC, with a corresponding lower percent of numbered citations while the percent of extra citations remains about the same in the 79 two journals. If the number of times a unique citation is repeated is not considered, then 40$ of the gross citations would have been ignored.

Table 3

Gross Citations Before Elimination

Citations Combined JACS JOC

numbered 1258 820 438 $ numbered 46.70$ 44.39$ 51.71$

extra-labelled 136 92 44 extra-unlabelled 198 139 59 $ extra 12.40$ 12.51$ 12.16$ unique 1592 1051 541 $ unique 59.10$ 56.90$ 63.87$ repeat 893 623 270 extra repeat 209 173 36 $ repeat 40.90$ 43.10$ 36.13$ gross 2694 1847 847

A separate data sheet was also kept for the source article listing the complete bibliographic information, the number of footnotes, the number of unique citations, a list of the citation numbers (superscripts), in order of appearance in the text with the three sections indicated, the number of repeat 80 citations and the gross citations (unique citations plus repeat citations). In addition, the length of the source article and of each section was measured to the nearest quarter column and recorded.

Once the source articles had been selected and the data sheets prepared for each citation to be studied, the rest of

the data were collected. The source article was read through twice, once for the general content, then in greater depth for detail. Each citation was taken in order of appearance in the source article and the actual cited article was obtained and read. With both the source article and the cited article in hand, a decision was then made as to exactly what was being cited from the cited article by the source article. If a relationship could not be determined between the two articles, or if the cited article could not be located from the biblio­ graphic citation given, the citation was counted as a non-fit citation.

Table 4 presents the figures for the combined data

(Table 5 for JACS and Table 6 for JOC) covering the elimina­ tion of the citations which were not or could not be studied. In each category, the first number listed is the number of unique bibliographic citations (in the case of the non-bibliographic category, the data refers to the corres- 81

Table 4 Gross Citations Studied by Section

Citations Total T/I R/D Exp before elimination unique -, ^ 1495 477 697 321 I088ra 209r 547r 332 r I07refb 4ref 90ref I3ref 4dc 2d 2d gross 2694 690 1336 668 non-bibliographic unique 223 76 95 52 41r 3r 22 r 16r 3 ref 2 ref lref gross 267 79 119 69 book unique 139 36 64 39 I06r 40r 56r lOr 35ref 2 ref 29*ef 4ref gross 280 78 149 53 thesis unique 25 14 7 4 llr lr 2r 8r gross 36 15 9 12 patent unique 9 2 4 3 3r 3r •*, 2 ref lref lref gross 14 2 5 7 non-fit unique 15 3 8 4 I04r llr 56r 37r 20ref I7ref 3ref Id Id gross 140 14 81 45 82

Table 4 (Continued)

Citations Total T/I R/D Exp citations studied • unique 1084 346 519 219 82 3r I54r 411r 258r 47ref 2 ref 41ref 4ref 3d 2d Id gross 1957 502 973 482

repeat citations

refer citations

'duplicate citations 83 Table 5

Gross Citations Studied from JACS by Section

Citations Total T/I R/D Exp before elimination unique 976 276 483 217 a 785r I70r 418r 197r 84refb 3ref 76ref 5ref 2dc Id Id gross 1847 449 978 420

1 non-bibliographic unique 123 37 60 26 26r 2r I6r 8r 2 ref 2 ref gross 151 39 78 34 book unique 102 30 45 27 lOlr 40r 54r 7r • 33ref 2ref 27ref 4ref gross 236 72 126 38 thesis unique 20 9 7 4 v llr lr 2r 8r gross 31 10 9 12 patent unique 1 1 gross 1 1 non-fit unique 7 1 3 3 78r llr 42r 25r 14ref Href gross 99 12 59 28 84 Table 5 (Continued)

Citations Total T/I R/D Exp citations studied unique 723 199 367 157 569r H6r 304r 149r 35ref lref 33ref lref 2d Id Id gross 1329 316 705 308

repeat citations refer citations duplicate citations 85

Table 6 Gross Citations Studied from JOC by Section

Citations Total T/I R/D Exp before elimination unique 519 201 214 104 303ra 39r l29r 135r 23refb lref 14ref 8ref 2dc Id Id gross 847 241 358 248 non-bibliographic unique 100 39 35 26 l5r lr 6r 8r lref lref gross 116 40 41 35 book unique 37 6 19 12 5r 2r 3r 2ref 2ref gross 44 6 23 15 thesis unique 5 5 gross 5 5 patent unique 8 3 3 3r 3r 2ref lref lref gross 13 4 7 86

Table 6 (Continued)

Citations Total T/I R/D Exp non-fit unique 8 2 5 1 26r 14r l2r 6ref 3ref 3ref Id Id gross 41 2 22 17 citations studied unique 361 147 152 62 254r 38r I07r 109r 12 ref lref 8ref 3ref Id Id gross 628 186 268 174

repeat citations

refer citations

"duplicate citations 87 ponding number of non-bibliographic citations), the next group of numbers are (1) the number of repeat citations (r), (2) the number of footnotes which refer to previously numbered citations (ref), and (3) the number of duplicates (d). The duplicate category indicates that the same bibliographic citation was listed under two different numbers in the same article. The final number listed in each category is the resulting gross number of citations (unique citations plus repeat, refer, and duplicate citations). From this stage on in the study, the repeat, refer, and duplicate citations are all classified together as repeat citations since, in each case, they do not represent new bibliographic citations.

In Table 7, the eliminated citations and the correspond­ ing percent of the citations before elimination are given in each category for both the unique citations and the gross citations, and for the combined data as well as for each source journal. About 25-30$ of both the unique citations and the gross citations were eliminated; thus the eliminated unique citations are repeated proportionately about the same as the unique citations studied. If the non-bibliographic citations and the non-fit citations are removed, then only

11.57$ of the unique citations before elimination and 12.25$ 88

Table 7

Eliminated Citations

Citations Combined JACS JOC non-bibliographic unique 223 123 100 $ 14.92 12.60 19.27 gross 267 151 116 $ 9.91 8.18 13.70 book unique 139 102 37 $ 9.30 10.45 7.13 gross 280 236 44 10.39 12.78 5.19 thesis unique 25 20 5 $ 1.67 2.05 0.96 gross 36 31 5 1.34 1.68 0.59 patent unique 9 1 8 $ 0.60 0.10 1.54 gross 14 1 13 $ O.52 0.05 1.53 non-fit unique 15 7 8 $ 1.00 O.72 1.54 gross 140 99 41 $ 5.20 5.36 4.84 total citations eliminated unique 411 253 158 $ 27.49 25.92 30.44 gross 737 518 219 27.36 28.05 25.85 89 of the gross citations before elimination are not studied because they cite literature other than journal articles.

Some points of difference can be noted between the two source journals. JACS tends to cite more books and more theses than JOC, while JOC tends to cite more patents and have more non-bibligraphic citations than JACS.

The non-bibliographic citations included several distinct categories. Those readily classified included: acknowledgement of financial assistance (including the identification of fellows), the author to whom inquiries should be addressed

(identification of the major author), unpublished data, an indi­ cation that a paper based on the work had been presented previously at a meeting, a series note (if not already included in the title), personal thanks for technical assistance, helpful conversations, commercial gifts of chemical compounds, authentic samples, etc., elaboration of general experimental procedures (specific machines used, uncorrected mp, etc.) and various types of explanatory statements.

Books were eliminated from the study since the primary concern was with the citing of material from journal articles.

In the case of the theses, many were bibliographic citations to the complete thesis upon which the article had been based. 90

Theses and patents were excluded because of their inaccessi- o bility for the present study.

The non-fit citations are those for which either a relationship to the source article could not be determined, the bibliographic data were so incorrect that the cited article could not be found, or the journal was not available in the University of Illinois Library. Table 8 presents the figures for the combined data and for each source journal.

Table 8

Non-fit Citations

Combined JACS JOC unique gross unique gross unique gross not in U of I Library 2 3 2 3 numerical 86 70 16 bibliographical 10 12 4 4 6 8 undetermined 3 39 1 22 2 17 total non-fit citations 15 140 7 99 8 41

The numerical non-fit citations are those which can be accounted for by the consideration that, for purposes of this 91

study, each bibliographic citation was given a unique number.

Thus, when several extra citations, either labeled or unlabeled, were listed under one footnote number, each time that number was repeated it was assumed that all the citations listed were being repeated. In many cases this was not so; it was obvious

that the repeat citation actually referred to only one (or more but not all) of the bibliographic citations listed under that number. For the repeat, undetermined, non-fit citations, it was felt in many cases that the wrong superscript had been given since the information was often found in another citation.

The bibliographical non-fit citations were due to totally incorrect bibliographic data.

It was considered important to see if the elimination of the non-bibliographic citations, the books, theses, and patents, and the non-fit citations had changed the overall distribution of the number of citations in the three sections.

It might be found that these eliminations had been predominately from one section and, therefore, the sample of citations to be studied had become numerically distorted with respect to the original distribution of citations in the three sections.

The number of unique citations and gross citations in each of the three sections before and after the elimination is presented in Table 9,for the combined data. A Chi Square 92

Table 9

Distribution of Citations by Section Before and After Elimination

Total T/I R/D Exp unique citations

before elimination 1495 477 697 321

studied 1084 346 519 219 (expected--on basis of distribution before elimina­ tion) (1084) (345.87) (505.38) (232.75) Chi Square =1.18

gross citations

before elimination 2694 690 1336 668

studied 1957 502 973 482 (expected--on basis of distribution before elimina­ tion) (1957) (501.24) (970.51) (485.25 Chi Square - 0.03 critical value (2 degrees of freedom, 0.05 level of significance) - 5.99 93 test was then performed on the data for the citations to be

studied, using the data for the distribution of the citations before elimination to determine the expected values for the distribution after elimination, in order to test the null hypothesis that the distribution of the citations among the three sections is not significantly different before and after the eliminations.

For the unique citations, a value of Chi Square =1.18 was obtained, not significant at the 0.05 level. Therefore the null hypothesis can not be rejected and it can be con­ cluded that the distribution of the unique citations among the three sections has not significantly changed with the eliminations. When the repeat citations are added, a value of

Chi Square = 0.03 is obtained for the gross citations, again not significant at the 0.05 level. The data were then separated for the two source journals and similar Chi Square tests were performed in Table 10 (JACS) and Table 11 (JOC).

Again the Chi Square values obtained are not significant at

1 the 0.05 level and the distribution of the unique citations and the gross citations with respect to the three sections has not been changed significantly by the elimination procedure.

It might be noted that the number of unique citations given in Table 4 (1495) does not agree with the number of 94 Table 10 Distribution of Citations from JACS by Section Before and After Elimination

Total T/I R/D Exp unique citations

before elimination 976 276 483 217

studied 723 199 367 157 (expected—on basis of distribution before elimina­ tion) (723) (204.45) (357.80) (160.75) Chi Square =0.48

gross citations

before elimination 1847 449 978 420

studied 1329 316 705 308 (expected--on basis of distribution before elimina­ tion) (1329) (323.08) (703.71) (302.21) Chi Square =0.27 critical value (2 degrees of freedom, 0.05 level of signifi cance) = 5.99 95 Table 11

Distribution of Citations from JOC by Section Before and After Elimination

Total T/I R/D Exp unique citations

before elimination 519 201 214 104

studied 361 147 152 62 (expected--on basis of distribution before elimina­ tion) (361) (139.81) (148.85) ( 72.34)

Chi Square = 1.92

gross citations

before elimination 847 241 358 248

studied 628 186 268 174

(expected--on basis of distribution before elimina­ tion) (628) (178.69) (265.43) (183.88

Chi Square =0.85 critical value (2 degrees of freedom, 0.05 level of signifi­ cance) = 5.99 $6 unique citations given in Table 3 (1592). This difference is due to the refer citations and the duplicate citations which were counted as unique citations in Table 3 but which are actually not new bibliographic citations and so are counted separately in Table 4. Similarly, the number of repeat citations does not agree (1102 in Table 3, 1088 in

Table 4) again due to the refer citations, since the repeat of a refer was counted as a refer citation in Table 4 but as a repeat citation in Table 3.

One reason for the existence of chis refer citation is that an explanatory statement might be desirable along with the bibliographic citation, but for only one of the uses of that citation and not all. In some cases, this was not done and as a result, repeat citations were made to footnotes which included explanatory statements which really did not apply in the particular case. This was not considered to be an error and the explanatory statement was ignored as necessary.

Since the actual cited article was looked at, rather than just the bibliographic information given in the source article, certain errors in the bibliographic data were found which would not have been discovered otherwise. A list of these errors was kept according to several categpries and is presented in

Table 12 for the unique citations. 97 Table 12

Unique Citation Errors

Combined JACS JOC

first author 41 30 11

other authors 34 20 14

j ournal 6 3 3

page 44 30 14

volume 3 2 1

year 7 1 6

total errors 135 86 49

$ errors 12.45 11.89 13.57

These categories include first named author, other authors

(the first named was separated because of its importance in bibliographies and citation indexes), journal title, page, volume, and year. The author errors included spelling errors, omission of names, and giving the names in the wrong order.

When the bibliographic citation did not lead to the first page of an article by the authors named, Chemical Abstracts was

consulted in order to obtain the correct bibliographic information and thereby the correct article. If the correct bibliographic 98 citation was not readily identified (e.g., if there were more than one article in the given year by the same authors), the citation was counted as a non-fit. A few journal errors occurred because of the similarity in names (e.g., Journal of

Physical Chemistry and Journal of Chemical Physics). Page errors were often the result of citing the exact page on which the cited information could be found (primarily experi­ mental data) rather than the first page of the article. This therefore, is not really an error since the cited information is readily found from the bibliographic citation; however, in a bibliography or a citation index it would be identified as a different entry than that for the article given in the standard manner, i.e., first page of the cited article. If more than one error appeared in a bibliographic citation, the error was counted in the first category as listed in Table 12. It can be seen that a bibliographic error appeared in over 10$ of the citations studied and again, this error is of the same magnitude for both of the source journals. This list of errors does not include the non-fit errors eliminated previously.

Since the problem of the repeat citation seems to be quite important, it was thought useful to determine how many of the unique citations had actually been repeated. This information is presented in Table 13 for the combined data 99 and for each of the source journals. As can be seen, in each case over one-third of the unique citations are repeated for an average of slightly over two times. Again, the figures for the two source journals are quite similar.

Table 13

Number of Unique Citations Repeated

Combined JACS JOC unique citations 1084 723 361 unique citations repeated 397 271 126 $ repeated 36.62 37-52 34.90 repeat citations 873 606 267 average number of repeat citations/unique cita­ tions repeated 2.20 2.24 2.12

Citation Level

Rather than measure the amount of cited information in a purely quantitative manner (e.g., by the number of lines of material), it was decided to use a qualitative measure so that the important factor actually becomes the,relative amount cited or the level rather than the absolute amount 100 cited. This also compensates for the varying lengths of the source articles and is another reason why the study was confined to journal articles, so that the range of the levels would be fairly narrow. The three levels of cited material identified were (1) the whole of the cited article, (2) a part of the cited article (sentence-paragraphs), and (3) specific words or data elements from the cited article (including numerical data, units of measurement, physical constants, etc.). Certain criteria were established so that the choice of level would at least be consistent throughout the study. The whole article was considered as having been cited if the source article, in some way, indicated that the citation in the source article was from a discussion of a specific topic which then was the primary topic of the cited article. The following examples give some indication of this whole article relationship: (the citing statement from the source article is given first, followed by the title of the corresponding cited article.) JACS-1226-1. "Several diagnostic tools designed to detect and measure them [quinones] have been applied and k refined including paper chromatography, thin layer chroma- 5 67 tography, ultra violet, 'infrared,' and nuclear magnetic resonance." 101

JACS-1226-1-4(A). Paper chromatography of vitamin K, and

related compounds.

JACS-l226-l-4(B). A rapid method for the identification of

small quantities of liquid-soluble vitamins and quinones

in biological material.

JACS-1226-1-5. Thin layer chromatography of naturally

occurring quinones and hydroquinones.

JACS-1226-1-6. The ultraviolet absorption of vitamin K,.

JACS-I226-I-7. The physiochemical characterization of vitamin

K homologues. I. Identification by a novel method of

quantitative IR spectrophotometry.

JACS-1226-1-8. Anwendung der Protonen resonanz zur Struk-

turaufklarung und identifizierung naturlich verkommender

un synthetischer chinone.

JACS-5820-2. "the kinetics of the oxidative cleavages J of ci-glycols by periodic acid have been studied extensively" 3'

JACS-5820-2-3(A), 3(B), 4(A), 4(B), 4(C). The mechanism of

oxidation of a-glycols by periodic acid. I-V.

JACS-2803-3. "in view of the well documented reactivity k of epoxides and epoxy ketones under a variety of conditions"

JACS-2803-3-4. Mechanisms of epoxide reactions. Q JACS-2803-3. "early studies by Tiffeneau7 must be con­

sidered" 102

JACS-2803-9(A), 9(B), 9(C), 9(D). All are by Tiffeneau.

Such statements as "have been studied extensively",

"early studies", "in their classical study" serve as an indication that the citation is to the whole article. How­ ever, their use alone is not sufficient; the decision could not be made from the source article alone since, in each case, the actual information cited could be (and often was) only a part of the cited article.

These examples also illustrate the inconsistency of the placing of the superscript. In a statement such as "in their recent study* X and Y* have shown that the reaction* of A with B produces C*" there are several possible places where the superscript might be placed, each producing a very slightly different emphasis, viz., emphasis on the study, on the author, on the reaction, on the product.

Also illustrated is the multiple citation, that is two or more superscripts appearing together. This was done in many cases since a later repeat citation was to be made to only one of the articles and therefore, all the bibliographic citations could not be listed under one number. In both cases, the extra citation and the multiple citation, it might be suspected that a closer than normal relationship exists between the two citations (a form of bibliographic 103

coupling). This might be an area for future investigation

into citation behavior.

A problem occurred when the citation was made in a

statement which indicated that the whole article was being cited (often indicated by the use of words such as "study"), but which was then followed by a statement which indicated

that not necessarily all of the study was of particular interest (usually indicated by "showed that"). If the

citation was repeated in the second statement, then the

first citation was considered to be to the "study", the second

to the "showed that". If the citation was not repeated, the two statements were considered as a unit and the "showed that" took preference. However, if there were several sentences discussing the "study" before the "showed that" statement, then the "study" took preference.

If the material being cited was found in several dif­ ferent places in the cited article, but was essentially the same information in each case, this was then counted as a part of the cited article. If the material cited was found as three or more large units in different places throughout the cited article, in such a way that the sum of the parts was necessary for the required information, then the whole article was counted. This might occur, for example, if a 104 series of related compounds were studied with respect to several topics in the cited article. If the source article were interested in only one topic for the series of compounds, or for all the topics but for only one of the compounds, then this information might be scattered throughout the cited article and would, in effect, necessitate the use of the whole article to locate the desired information.

In many cases, experimental details were discussed in the results/discussion section as well as given in the experimental section of the cited article. Since cited material found in the experimental section of the cited article was to be noted for later use, some decision had to be made as to which section had preference. For purposes of this study, any material found in the experimental section of the cited article was noted, whether or not the material could also be found in part or as a whole in the results/ discussion section.

Any explanatory statement (non-bibliographic citation) given in the footnote was considered if necessary in making the decision as to the amount of material being cited. In fact, in some cases, the reason for the citation was meaning­ less if only the text was considered. This is an area for future study since most citation-count studies (as in this 105 one) make no distinction between this type of footnote citation and the regular text citation. Perhaps they will have different properties or can be used in some manner to give some indication of the probable relevance of the citation to a certain problem.

A number of these explanatory statements indicate an acknowledgement of the priority which has been established in the literature by some other scientist.

nt JACS-1226-1. "to give the fragments, p, q, r, s, and t" '

JACS-1226-1-22. "Essentially the same fragmentation has been postulated for this compound ... in a paper by . . . which appeared after submission of our report."

JOC-436-3. "the conclusion . . . that cis-10-methy1-2- 9a decalone has the 'non steroid' conformation7 " JOC-436-3-9(a). "Note added in proof . . . have recently

reported the use of essentially the same nmr method as

used in this paper to arrive at the same conclusion

concerning the conformation."

Another use of these explanatory statements was to indicate analogous or related type of behavior as reported in the literature. Without the statement indicating the use of analogy, it might not be exactly clear why the article was being cited. 106

Citations to the whole article were readily determined, as were citations to specific data or words and to experimental procedures measured as parts. Most citations to a part of the cited article were also quite obvious, and in the few difficult cases, a process of elimination was used. 107

Footnotes

American Chemical Society. Handbook for Authors of Papers in the Journals of the American Chemical Society. Washington, D. C, American Chemical Society Publications, I967.

William H. Beyer, ed. CRC Handbook of Tables for Probability and Statistics. Cleveland, Chemical Rubber Co., 1966. pp. 341-345. 108

RESULTS AND DISCUSSION

The following distributions of the citations chosen for investigation were studied with respect to the three sections of the source journal articles reporting the results of original research in the field of organic chemistry (the title/introduction (T/I), the results/discussion (R/D), and the experimental (EXP)) (1) the frequency distribution, .(2) the citation level distribution, (3) the self citation and the same journal citation distribution, (4) the journal dis­ tribution, and (5) the time distribution. The first three studies were conducted in order to investigate the specific hypothesis, that the amount of material actually cited will vary significantly with the section of the source article in which the citation occurs, viz., that citations in the T/I will tend to be to the whole of the cited article, citations in the R/D will tend to be to a part of the cited article, and citations in the Exp will tend to be to word/words of the cited article.

In addition, in these first three studies, the data were separated for the two source journals in order to see if differences in citation habits might exist with respect to the journal to which the source article was submitted for publication. 109

The last two studies were conducted in order to give some

indication whether the properties of citations in general

will tend to vary with the section of the source article in

which the citation appears.

In all five studies, and particularly in the first, the

number of unique citations (the number of bibliographic citations)

and the number of gross citations (unique citations plus repeat

citatipns) were considered separately, in order to illustrate

the importance of the repeat citations and the bias which can be introducted if they are ignored.

Frequency Distribution

The first property of the citations to be investigated was the number of citations and how they were distributed in

the source articles with respect to the three sections. These

data are presented in Table 14 for the number of unique cita­

tions and the number of gross citations by section for the combined sample from both source journals.

A Chi Square test was then performed on the frequency distribution of gross citations, using the percentages obtained from the corresponding frequency distribution of unique cita­ tions to determine the expected values, in order to test the null hypothesis that the frequency distribution of the citations 110

Table 14

Frequency Distribution of Citations by Section

Total T/I R/D Exp unique citations 1084 346 519 219 $ 31.92 47.88 20.20 gross citations 1957 502 973 482 $ 25.65 49.72 24.63 (expected—on the basis of distribution of unique citations) (1957) (624.68) (937-01) (395-31)

Chi Square = 44.48 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9.21

among the three sections is the same for the unique citations and the gross citations. A-value of Chi Square = 44.48 was obtained, significant at the 0.01 level; thus the null hypothesis can be rejected and it can be concluded that the two distributions are significantly different. Therefore, to count each citation only once in each article, the first time it is used, and not take into account the number of times the citation is repeated would give a significantly different frequency distribution pattern. Inspection reveals that the percentage of citations from the T/I decreases from unique citations to gross citations, Ill while the percentages from both the R/D and the Exp increase.

This is somewhat to be expected from the order of the sections, since the further on in the article, the more citations there are which can be repeated.

The percentage of gross citations, in the T/l and the

Exp, are about the same and each about one-half of the R/D

(25.65$, 24.63$, and 49.72$, respectively).

The data were then separated according to source journal in Table 15 (JACS) and Table 16 (JOC) to see if there were variations in the frequency distribution according to section in the two source journals.

Table I5

Frequency Distribution of Citations from JACS by Section

Total T/I R/D Exp unique citations 723 199 367 I57 $ 27-52 50.76 21.72 gross citations 1329 316 705 308 $ 23.78 53.05 23.17 (expected--on the basis of distribution of unique citations) (1329) (365-74) (674.60) (288.66)

Chi Square = 9.43 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9.21 112

Table 16

Frequency Distribution of Citations from JOC by Section

Total T/I R/D Exp unique citations 36I 147 152 62 $ 40.72 42.11 17.17 gross citations 628 186 268 I74 $ 29.62 42.67 27.71 (expected--on the basis of distribution of unique citations) (628) (255.72) (264.45) (I07.83) Chi Square = 59.67 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9 .21

The most noticeable difference is that there are just about exactly twice as many citations from the twenty-five JACS articles as there are from the twenty-five JOC articles, both for unique citations (2.00 times) and for gross citations

(2.11 times). That this factor remains more or less constant also indicates that the increase in the number of citations from unique citations to gross citations is proportionately about the same for each journal. There are 1.84 times as many gross citations as unique,citations for JACS, 1.74 times for

JOC. Therefore, while JACS has twice as many citations as JOC 113

from the same number of source articles, both journals behave

in a similar manner with respect to the repeating of citations.

Again looking at the frequency distribution of citations

according to section and comparing frequency distributions of

the unique citations and the gross citations for JACS, Chi-

Square = 9.43 and for JOC, Chi Square = 59.67, both significant

at the 0.01 level. Therefore the null hypothesis that the

frequency distribution does not change can again be rejected,

and it can be concluded that the frequency distribution of unique citations and gross citations are significantly different

in both source journals. While both are significant at the 0.01

level, the larger Chi Square value for JOC indicates that the

change in frequency distribution from unique citations to gross citations, is greater for JOC than JACS. In both source journals there is a similar drop in percentage in the T/I and

an increase in the R/D and the Exp. Also, in both source journals, the percentage of gross citations is approximately

the same in the T/l and Exp (23-78$ vs 23-17$ for JACS;

29-62$ vs 27.71$ for JOC). However, in JACS these percentages are lower than those in JOC, with a corresponding higher per­ centage in the R/D of JACS. This might indicate that a greater emphasis is placed on the R/D section in articles appearing in JACS than JOC or the corollary, that there is more emphasis 114

placed on the T/I and Exp in articles appearing in JOC than

JACS.

The total number of citations and the frequency distribu­

tion of the citations throughout the source articles by section

can not be examined without, at some point, taking into con­

sideration the length of the source article or the amount of

material in each section in which the citations occur. This

information is presented in Table 17 where the length of the

source article is given in the number of columns to the

nearest one-quarter column (where one page consists of two

columns of material). Both JACS and JOC have exactly the

same format and so the lengths for each source article can

be added directly. However, a correction factor is needed

for the Exp since smaller type is used. This correction

factor was calculated as follows:

65 lines of large type = 80 lines of small type (one column) 10 lines of large type = 11 lines of small type (number of words)

correction factor = (80 + 65/l0)/65 = 1.33

The number of columns in the Exp was therefore multiplied

by 1.33 atid then rounded to the nearest quarter of a column

so that the lengths (amount) of material for the sections could

0be compared. The percentage of the total length of material 115 was also calculated for each section. The concentration of citations (number of citations/column) was then determined for the total and for each section.

Table 17

Concentration of Citations by Section

Total T/I R/D Exp number of columns 581.25 65.75 361-50 154.00 number of columns corrected3 632.00 65.75 36I.5O 204.75

$ 10.40 57.20 32.40 unique citations 1084 346 5I9 219 unique citations/column I.72 5.26 1,44 I.07 gross citations 1957 502 973 482 gross citations/column 3.10 7.63 2.69 2.35

correction factor = 1.33 x Exp

As can be seen in Table I7 for the unique citations, the

T/I concentration is the highest (5.26) and is considerably higher than the concentrations for the R/D (1.44) and the Exp

(1.07), which are quite close in value. Since the total column 116 length of the source material does not change, the concen­ trations must increase from unique citations to gross cita­ tions; however, the order of the concentrations in each section remains approximately the same.

When the data for the individual source journals are separated in Table 18 (JACS) and Table 19 (JOC) there are some differences. Most important is the fact that the larger number of citations in JACS can not be entirely explained by the greater length of source material (352.25 columns JACS;

280.00 columns JOC) as can be seen by comparison of the con­ centrations for the total unique citations and gross citations

(2.05 JACS; 1.29 JOC: 3-77 JACS; 2.24 JOC respectively).

Therefore it can be concluded that there is, on the average, a larger concentration of citations in the articles of JACS than in the articles of JOC.

In both source journals, the T/I has the largest concen­ tration of citations, due primarily to the small amount of source material from which the citations are made. This high concentration is to be expected since the primary purpose of the introduction is to discuss previously published related work, all of which should be cited. If the few citations from the title (related papers by the same author, usually in the same series) and the corresponding columns of material 117

Table 18

Concentration of Citations from JACS by Section

Total T/I R/D Exp number of columns 330.25 37.00 226.75 66.50 number of columns corrected3 352.25 37.00 226.75 88.50

10.50 64.37 25-13 unique citations 723 199 367 157 unique citations/column 2.05 5-38 1.62 1-77 gross citations 1329 316 705 308 gross citation/column 3.77 8.54 3.11 3-48

correction factor = 1.33 x Exp

devoted to the title, authors' names, affi-liation, and abstract were subtracted, the concentration for the introduction would be even higher.

It can also be seen that, while the percentage of T/l material is about the same for the two source journals (10.50$

JACS; 10.27$ JOC), JOC has not only a larger percentage of material in the Exp (41.61$ JOC; 25.13$ JACS) but also has, in fact, a larger amount of material (87.50 columns JOC; 118

Table 19 Concentration of Citations from JOC by Section

Total T/I T/D Exp number of columns 251.00 28.75 134.75 87.50 number of columns corrected3 280.00 28.75 134.75 116.50

$ 10.27 58.12 41.61 unique citations 361 147 152 62 unique citations/column 1.29 5.11 1.13 0.53 gross citations 628 186 268 174 gross citations/column 2.24 6.47 1.99 1.49

correction factor = 1.33 x Exp

66.50 columns JACS). As a result, the largest differences between the section concentrations for JACS and JOC occur in the Exp. Although JACS has a larger number of citations in the R/D than in the Exp, the corresponding larger amount of material in the R/D results in a somewhat smaller concentration for the R/D than for the Exp (I.62 R/D, I.77 Exp, unique citations; 3.11 RA/,3.48 Exp, gross citations). 119

From these considerations, it might be concluded that the two source journals have a slightly different emphasis,

JACS with both a larger percentage of citations, and a larger percentage of material in the R/D appears to emphasize presentation of the results and discussion of their implica­ tions, while the larger amount of material in the Exp of JOC might indicate the use of JOC by the organic chemist to present results which need less interpretation.

Citation Level Distribution

The major concern of this study was to determine whether or not citations can be treated as equivalent or similar with respect to the level (or amount) of information that is being cited. The hypothesis being tested is that, although the citation level does vary, it does so in a predictable manner; citations from the T/l of the source article will tend to cite large amounts of material (the whole of the cited article), citations from the R/D of the source article will tend to cite smaller amounts of material (sentence-paragraphs of the cited article), and citations from the Exp will tend to cite small bits of material (word/words from the cited article).

The rationale behind this hypothesis is that the T/I will tend to cite an article which deals more or less entirely 120 with a topic broad'y related to the source article. The Exp on the other hand might be expected to cite very specific pieces of information related primarily to the specific chemical compounds which are synthesized or used in the research,

information which could therefore be found in articles dealing not at all with the same topic as the source article. The citations from the R/D might be expected to be more evenly distributed among the three levels since the discussion of the research could range from the topic of the research as a whole to the preparation or identification of specific chemical compounds. However, it is suggested that the discussion of the various steps involved in the research would lead to citing predominately parts of cited articles related to these individual steps in the research, articles which may not deal entirely with the same general topic of research as the source article, but which are related in part.

The procedure followed in determining the three levels used in this study (the whole, part, or word/words from the cited article) has been presented in the preceding chapter.

The results for the combined sample of unique citations from both source journals are shown in Table 20 with the expected values (on the basis of column and row totals) given in parentheses. 121 Table 20 Citation Level Distribution (Whole-Part-Word/Words) of Unique Citations by Section

Total T/I R/g Exp whole 456 261 181 14 (145.55) (218.32) (92.13) part 485 75 274 I36 (154.81) (232.21) (97.98) word/words 143 10 64 69 (45-64 (68.47) (28.89) unique citations 1084 346 5I9 219 1 Chi Square = 311.43 critical value (4 degrees of freedom, 0.01 level of signifi­ cance) = 13.28

A Chi Square test was then performed to test the null hypothesis that there is no relationship between the level of material being cited and the section of the source article in which the citation occurs. Using the values obtained in this study, Chi Square = 311.43, significant at the 0.01 level. Therefore, the null hypothesis can be rejected and it can be concluded that a significant relationship does exist between the citation level and the section of the source article in which the citation occurs. 122

Inspection reveals that the expected trend was obtained

for the T/I and the R/D, but that the Exp had more citations

to parts than to word/words, contrary to expectations. How- » ever, many of the citations to parts of the cited articles

were to a paragraph of synthesis or identification of a

specific chemical compound, material which appeared in the

experimental section of the cited article. Since a citation to

a physical constant for a specific chemical compound and a

citation to the synthesis of that same chemical compound seem

to be similar in nature (and present a similar indexing problem

in that both could be indexed, and retrieved, by a heading such

as: chemical compound - attribute), these parts from the

experimental section of the cited article were added to the

word/words category to form a new level labeled word/words*.

These data are then presented in Table 21.

A Chi Square test was again performed and produced a value

of Chi Square = 666.18, again significant at the 0.01 level

(and over twice the value previously obtained). Inspection now reveals that the trend is just as predicted with therT/I having the largest number of its citations to the whole of the

cited article, the R/D to parts of the cited article, and the

Exp to word/words* (all citations to word/words plus citations

to parts from the experimental section) from the cited article. 123 Table 21

Citation Level Distribution (Whole-Part-Word/Words*) of Unique Citations by Section

Total T/I R/D Exp whole 456 261 181 14 (145-55) (218.32) (92.13) part 344 73 256 15 (109.80) (164.70) (69.50) word/words* 284 12 82 190 (90.65) (135.98) (57-37) unique citations 1084 346 519 219 Chi Square = 666.18 critical value (4 degiree s of freedom, 0.01 level of signifi- cance) = 13.28

When the repeat citations are added in Table 22, and Table 23, a similar pattern is obtained with Chi Square = 532.75 in Table 22 (whole-part-word/words distribution) and Chi Square = 1173.97 (again over twice the value previously obtained) in Table 23 (whole-part-word/words* distribution). The very large Chi Square values are, of course, in part due to the large sample size; however, the large sample was necessary in order to obtain sufficient values for each cell of the contingency table. 124

Table 22

Citation Level Distribution (Whole-Part-Word/Words) of Gross Citations by Section

Total T/I R/D Exp

whole 588 324 246 18 (150.83) (292.35) (144.82)

part 1023 164 573 286 (262.42) (508.62) (251.96)

word/words 346 14 154 I78 (88.75) (172.03) (85.22)

gross citations I957 502 973 482

Chi Square = 532.75

critical value (4 degrees of freedom, 0.01 level of signifi­ cance) = 13.28

The large Chi Square values are obtained from the expected

predicted higher concentrations, and also from the much lower

than expected values for the T/l-word/words (and word/words*)

cell and the Exp-whole cells; very few citations from the T/I

are to small amounts of data, very few citations from the Exp

are to whole articles. The R/D column contributes less to

the Chi Square value than the other two sections since the distri bution of the citations among the three levels is less extreme.

As mentioned previously, this is somewhat to be expected 125 Table 23 Citation Level Distribution (Whole-Part-Word/Words*) of Gross Citations by Section

Total T/I R/D Exp whole 588 324 245 18 (150.83) (292.35) (144.82) part 721 156 534 31 (184.95) (358.47) (177-58) word/words* 648 22 193 433 (166.22) (322.18) (159.60) gross citations 1957 502 973 482 Chi Square = II73.97 critical value (4 degrees of freedom, 0.01 level of signifi­ cance) = 13.28

since the R/D could range from discussion of the whole research problem to synthesis and identification of the specific chemical compounds involved. However, there are still many more cita­ tions to parts than the expected values. The data for the two source journals were then separated and Chi Square tests again performed to test the null hypothesis that there is no relationship between the level of material cited and the section of the source article in which the citation occurs. The values obtained are given in Table 24 (JACS) 126

Table 24

Citation Level Distribution of Citations from JACS by Section

Total' ' T/I R/D Exp

whole 284 150 124 10 part 326 45 185 96 word/words 113 4 58 51 unique citations 723 199 367 157 Chi Square = 197.84 whole 362 191 158 13 part 707 117 412 178 word/words 260 8 135 117 gross citations 1329 316 705 308 Chi Square = 314.62 whole 284 150 124 10 part 237 43 184 10 word/words* 202 6 59 137 unique citations 723 199 367 157 Chi Square = 447.36 whole 362 191 I58 13 part 544 113 408 23 word/words* 423 12 139 272 gross citations 1329 316 705 308 Chi Square = 748.86 critical value (4 degrees of freedom, 0.01 level of signifi­ cance) = 13.28 127

Table 25

Citation Level Distribution of Citations from JOC by Section

Total T/I R/D Exp whole 172 111 57 4 part 159 30 89 40 word/words 30 6 6 18 unique citations 361 147 152 62 Chi Square = 119.82 whole 226 133 88 5 part 316 47 161 108 word/words 86 6 19 61 gross citations 628 186 268 174 Chi Square = 233-39 whole 172 111 57 4 part 107 30 72 5 word/words* 82 6 23 53 unique citations 361 147 152 62 Chi Square = 213.30 whole 226 133 88 5 part 177 43 126 8 word/words* 225 10 54 161 gross citations 628 186 268 174 Chi Square = 411.37 critical value (4 degrees of freedom, 0.01 level of signifi­ cance) = I3.28 128 and Table 25 (JOC) and all are significant at the 0.01 level.

Thus the null hypothesis can be rejected and it can be concluded that there is a significant relationship in each of the source journals. Table 24 and Table 25 also illustrate the additive power of Chi Square, where the sum of the Chi Square for each of the source journals is very close to the corresponding Chi

Square for the combined sample (e.g., for gross citations, whole-part-word/words*distribution, Chi Square = 1173.97 for the combined sample; Chi Square = 748.86 + 411.37 = 1160.23 for the sum).

Since the addition of the repeat citations produces a much larger value for Chi Square in both cases (word/words and word/words*), the distribution of the number of citations in each cell of the contingency table must be changing. There­ fore, the percentage distribution of the citation level with respect to section was determined for the unique citations and gross citations. These data are presented in Table 26 (whole- part-word/words distribution) and in Table 27 (whole-part-word/ words* distribution). For the total, and in each section, the addition of the repeat citations produces a decrease in the percentage of citations to the whole article in all cases, an increase in the percentage of citations to parts (except for the Exp), and an increase in the percentage of citations 129 Table 26 Percent Citation Level Distribution (Whole-Part-Word/Words) by Section

Total T/I R/D Exp whole $ unique citations 42.07 75.43 34.88 6.39 $ gross citations 30.05 64.54 25.28 3.73 part $ unique citations 44.74 21.68 52.79 62.10 $ gross citations 52.27 32.67 58.89 59.34 word/words $ unique citations 13.19 2.89 12.33 31.51 $ gross citations 17.68 2.79 15.83 36.93

t \

Table 27 Percent Citation Level Distribution (Whole-Part-Word/Words*) by Section

Total T/I R/D Exp whole $ unique citations 42.07 75.43 34.88 6.39 $ gross citations 30.05 64.54 25.28 3.73 part $ unique citations 31.73 21.10 49.32 6.85 $ gross citations 36.84 31.08 54.88 6.43 word/words * $ unique citations 26.20 3.47 15.80 86.76 $ gross citations 33.11 4.38 19.84 89.84 130 to word/words and word/words* (except for the T/l). The largest percentage of the repeat citations is to parts of cited articles (except for Exp-word/words*).

The reason for these changes in distribution can be seen more clearly in Table 28 (whole-part-word/words distribution) and Tabie 29 (whole-part-word/words* distribution), where the distribution of only the repeat citations is presented.

It was expected that the repeats would be predominately to parts and words of cited articles, rather than to the whole of the cited article. This is, of course, again related to the fact that the T/l, with its (shown) tendency to cite the whole article, appears first in the source article and there­ fore these citations are the ones counted as unique citations.

These data, of course, do not indicate which citations are being repeated. It is suggested that some of the unique citations which are later repeated are citations originally made in the T/l to whole articles with the idea that, if the article is pertinent enough as a whole to be cited, then parts of the cited article will also be pertinent in later discussions of smaller segments of the research. Thus, it might be suggested that, the greater the. number of citations to a cited article, the more important that article is, or the closer the relationship to the source article. However, 131

Table 28

Citation Level Distribution (Whole-Part-Word/Words) of Repeat Citations by Section

Total T/I R/D Exp whole 132 63 65 4 $ 40.38 14.32 1.52 part 538 89 299 150 $ 61.63 57.05 65.86 57.03 word/words 203 4 90 109 $ 23.25 2.55 19.82 41.45 repeat citations 873 I56 454 263

a Table 29 Citation Level Distribution (Whole-Part-Word/Words*) of Repeat Citations by Section

Total T/I R/D Exp whole 132 63 65 $ 15.12 40.38 14.32 .1.52 part 377 S3 278 16 43.18 53.21 61.23 6.08 word/words* 364 . 10 111 243 $ 41.70 6.41 24.45 92.40 repeat citations 873 I56 454 263 132 another source for repeat citations would be citations from

the R/D dealing with experimental procedures which are later repeated in the Exp. These would then be more or less identical citations and so would not indicate any additional measure of relatedness. This whole area of the repeat cita­ tion needs greater recognition and more study.

Self Citation and Same Journal Citation Distribution

The phenomenon of the self citation is frequently mentioned in citation studies as a probable confounding variable. There­ fore, the data were examined to see if the distribution of self citations was related to the section of the source article or changed with the addition of the repeat citations.

The combined data for the unique citations are presented in Table 30. The expected values on the basis of column and row totals are given in parentheses. A Chi Square test was then performed in order to test the null hypothesis that the distribution of the self citations is not related to the section of the source article in which the citation appeared.

A value of Chi Square = 25.41 was obtained, significant at the 0.01 level. Therefore, the null hypothesis can be rejected and it can be concluded that the distribution is related significantly to the section in which the citation 133

Table 30

Self Citation Distribution of Unique Citations by Section

Total T/I R/D Exp unique self citations 200 93 81 26 (63.84) (95.76) (40.40) unique non-self citations 884 253 438 193 (282.16) (423.24) (168.60) unique citations 1084 346 519 219 Chi Square = 25-41 critical value (2 degrees of freedom, 0.01 level of signi­ ficance) = 9.21

appeared. Inspection of the data reveals that the large Chi

Square value is due primarily to the larger than expected number of self citations in the T/l and somewhat less to the smaller than expected number in the Exp.

The corresponding data for the combined sample of gross citations are presented in Table 31 and a value of Chi Square

=0.34 was obtained. This is not significant at the 0.01 level (and is, in fact, not significant at the 0.80 level), the null hypothesis can not be rejected, and it can be con­ cluded that the distribution of self citations (as measured by gross citations) is not related to the section in which the citation appears. 134

Table 31

Self Citation Distribution of Gross Citations by Section

Total T/I R/D Exp gross self citations 507 132 255 120 (130.05) (252.08) (124.87) gross non-self citations 1450 370 718 362 (371.95) (720.92) (357.13) gross citations 1957 502 973 482 Chi Square = 0.34 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9.21

The significant Chi Square value obtained for the unique citations with a larger than expected number of self citations in the first section of the source article, the T/l, and a smaller than expected number in the last section, the Exp, followed by a very low value when the repeat citations are added, would indicate that the author does tend to cite himself proportionately throughout the whole source article. If the repeat citations are not taken into account, a false self cita­ tion distribution pattern is obtained, placing emphasis on the

T/I only because it comes first in the article. This even self citation distribution is to be expected since most researchers continue to work in specific areas of interest 135

and therefore self citations are most likely to be articles

dealing with a topic closely related to that of the source

article. An article dealing entirely with a closely related

topic is then likely to be cited several times, for different purposes, throughout the source article.

A similar type of citation, also often considered along with the self citation, is that of citation to the same journal

(i£., JACS to JACS; JOC to JOC). These data are presented in

Table 32 (unique citations) and Table 33 (gross citations).

In the case of the same journal citation, neither Chi Square value was significant at the 0.01 level, therefore it can be concluded that the distribution of the same journal citations is not significantlyNrelated to the section in which the citation occurs.

The data for the two source journals and the results of the Chi square tests are presented in Table 34 (JACS) and Table 35 (JOC).

Just as with the combined data, the Chi Square values indicate that a significant relationship exists between unique self citations and the section of the source article at the 0.01 level of significance, but not when the repeat citations are added. Thus both journals are behaving in a similar manner with respect to the self citation distribution with self citations being proportionately distributed throughout the 136

Table 32

Same Journal Citation Distribution of Unique Citations by Section

Total T/I R/D Exp unique same journal citations 334 106 172 56 (106.61) (159.91) ( 67.48) unique non-same journal citations 750 240 347 163 (239.39) (359.09) (151.52) unique citations 1084 346 519 219 Chi Square =4.14 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9.21

Table 33

Same Journal Citation Distribution of Gross Citations by Section

Total T/I R/D Exp gross same journal citations 662 150 353 159 (169.81) (329.14) (163.05) gross non-same journal citations 1294 352 620 323 (332.19) (643.86) (318.95) gross citations I957 502 973 482 Chi Square = 6.25 critical value (2 degrees of freedom, 0.01 level of signifi­ cance) = 9-21 137 Table 34

Self Citation Distribution and Same Journal Citation Distribution of Citations from JACS by Section

Total T/I R/D Exp Chi Square self citations unique 129 52 54 23 12.86 gross 338 75 179 84 1.03 same journal citations unique 288 84 153 51 4.54 gross 557 119 303 135 3.14 critical value (2 degrees of freedom, 0.01 level of signifi- cance) =9.21

Table 35 Self Citation Distribution and Same Journal Citation Distribution of Citations from JOC by Section

Total T/I R/D Exp Chi Square self citations unique 71 41 27 3 15.28 gross 169 57 76 36 5.04 same journal citations unique 46 22 19 5 1.88 gross 105 31 50 24 1.79 critical value (2 degrees of freedom, 0.01 level of signifi- cance) =9.21 136 Toole 3»

St Citation Distribution of Citations by Section

T/I R/D Exp unique sew citations 33% 106 172 56 (106.61) (159-91) ( 67.48) unique non-seme journal citations 750 2*0 3*7 163 (239.39) (359.09) (151.52) unique citations 3*6 519 219 Chi Square - *.l* critical veloe (2 of freedom, 0.01 level of signifi* cance) - 9«2t

Table 33 Citetioo Distribution of Citations by Section

T/I R/D Exp gross s citations 66t 150 353 159 (169.81) (329. U) (163.05) gross non journal 620 323 (332.19) (6*3.86) (318.95) gross citetines 973 482 Chi Square * 6 .#3 critical valme If of freedom, 0.01 level of signifi- cance) * 9-21 138 sections of the source article when gross citations are con­ sidered but concentrated in the T/I when only unique citations are considered.

Just how similar the two source journals are behaving with respect to self citation distribution can be seen in Table 36 where the percentage of self citations for unique citations, gross citations, and the repeat citations alone are presented for the combined data and for each source journal.

Table 36

Percent Self Citations of Unique, Gross, and Repeat Citations by Section

Combined JACS JOC unique self citations 200 129 71 18.45 17.84 19.39 gross self citations 507 338 I69 % 25.91 25.43 26.91 repeat self citations 307 209 98 35.16 34.49 36.70

The percentages obtained for JACS and JOC are very close to each other and therefore very close to the value for the combined data. For the combined data, the percentage of unique self citations is 18.45$, the percentage of gross self citations 139

is 25.91$, and the percentage of the repeat self citations

alone is 35.16$. Thus, the work that the author is familiar with and which is so related to his research that several

citations are made to it at different places in the source

article, is his own in 35$ of the time. In addition, there

is a difference of 7$ between the percentages of unique self

citations and gross self citations which would suggest that

citations studies which do not take the repeat citations into

account produce low values for the total amount of self

citation.

In Table 34 and Table 35, the Chi Square values obtained

indicate that there is no significant relationship between

same journal citations and the section of the source article

in which the citation appears, for both unique citations and gross citations.

The percentage of citations from the two source journals to each of the two source journals was also determined in order to see if there was a tendency for one journal to cite itself more than another journal does. From the results pre­ sented in Table 37, it can be seen that the percentage of total citations from JACS to JACS is greater than the per­ centage of total citations from JOC to JACS (39.83$: 34.35$ unique citations; 41.91$: 33.92$ gross citations respectively). 140

Table 37

Citations from JACS/JOC to JACS/JOC by Section

Total T/I R/D Exp unique citations to JACS ' from* JACS 288 84 153 51 $ 39.83 42.21 41.69 32.48 from JOC 124 42 62 20 34.35 28.57 40.79 32.26 to JOC from JACS 54 7 25 22 $ 7.47 3.52 6.81 14.01 from JOC 46 22 19 5 12.74 14.97 12.50 8.06 gross citations to JACS from JACS 557 119 303 135 $ 41.91 37.66 42.98 43.83 from JOC 213 55 93 65 $ 33.92 29.57 34.70 37.36 to JOC from JACS 93 11 33 44 $ 7.00 3.48 5.39 14.29 from JOC 105 31 50 24 .. I6.72 I6.67 18.66 13.79 141

Similarly, in the case of citations to JOC, the percentage of citations from JOC to JOC is greater than the percentage of citations from JACS to JOC (12.74$ : 7.47$ unique citations;

16.72$ : 7.00$ gross citations respectively). These relation­ ships are the same for each section as well, except for citations to JOC from the Exp. Thus, there does appear to be some tendency towards same journal citation beyond the value of the journal to the field.

In addition, for the total citations, the percentages of same journal citations increase from unique citations to gross citations (39.83$ to 41.91$ JACS; 12.74$ to 16.72$ JOC) while the percentages of citations from JACS to JOC and JOC to

JACS decrease very slightly at the same time. (However, this does not always occur when each section is inspected separately.)

It might also be noted that the percentage of citation from

JACS to JOC is highest in the Exp while for JOC to JOC the percentage is lowest in the Exp. This might indicate that

JACS articles use JOC material primarily for experimental purposes, while JOC articles tend to use other JOC articles for all purposes, which supports the idea of the more experimental nature of JOC compared to JACS. 142

Journal Distribution

One property of citations that is frequently investigated

is the distribution of the citations among the various journals,

usually by arranging the journals in order from the most cited

to the least cited. These lists of the most cited journals

in a particular field are often considered as indicating the

most important journals for that field.

Therefore, it was decided to look at the journal distri­

bution for the three sections in several different ways to see

if any differences did exist between the three sections (or

with the addition of the repeat citations) as an indication

that properties of citations in general tend to vary with

the section of the source article in which the citation occurs.

Rather than repeating the journal title each tiraej the journals were all given reference numbers. These reference numbers are listed in Table 38 beside the corresponding journal title. The journals are listed in the order in which they would have appeared in the usual citation study, the order for total unique citations. When more than one journal had the same number of citations, the journals were listed in alphabetical order. The specific number of citations for each journal will be found in Table 39. The form of the journal has been taken from Chemical Abstracts and is the 143 Table 38

Reference Numbers for Journals Cited

1. Journal of the American Chemical Society 2. Journal of Organic Chemistry 3. Journal of the Chemical Society 4. Chemische Berichte 5. Tetrahedron Letters 6. Tetrahedron 7. Annalen der Chemie, Justus Liebigs 8. Chemistry & Industry (London) 9. Journal of Chemical Physics 10. Recueil des Travaux Chimiques des Pays-Bas 11. Journal of Physical Chemistry 12. Bulletin de la Societe Chimique de France 13. Acta Chemica Scandinavica 14. Canadian Journal of Chemistry 15. Helvetica Chimica Acta 16. Proceedings of the Chemical Society 17. Nature 18. Chemical Abstracts 19. Comptes Rendus 20. Transactions of the Faraday Society 21. Angewandte Chemie 22. Chemical & Pharmaceutical Bulletin (Tokyo) 23. Science 24. Analytical Chemistry 25. Journal of Biological Chemistry 26. Zhurnal Obshchei Khimii 27. Collection of Czechoslovak Chemical Communications 28. Proceedings of the Royal Society (London), Series A 29* Australian Journal of Chemistry 30. Biochemical Journal 31. Biochimica et Biophysica Acta 32. Chemical Communications 33. Chemical Reviews 34. Experientia 35. Izvestiya Akademii Nauk SSSR, Otdelenie Khimicheskikh Nauk 36. Monatshefte fuer Chemie und Verwandte Telle Anderer Wissenschaften 37. Arkiv for Kemi 38. Bulletin of the Chemical Society of Japan; 39> Journal fuer Praktische Chemie 40. Journal of Chemical Education 144 Table 38 (Continued) 41. Molecular Physics 42. Progress in Organic Chemistry 43. Quarterly Reviews (London) 44. Radiation Research 45. Zeitschrift fuer Elektrochemie 46. Abstracts of Papers, ACS National Meeting 47. Archiv der Pharmazie 48. Bulletin des Societes Chimiques Beiges 49. Chimia (Aarau) 50. Chimica e l1Industrie (Milan) 51. Discussions of the Faraday Society 52. Doklady Akademii Nauk SSSR 53. Industrial and Engineering Chemistry 54. Industrie Chimique Beige 55. Israel Journal of Chemistry 56. Journal of Medicinal Chemistry 57. Philosophical Magazine 58. Photochemistry and Photobiology 59. Progress in Physical Organic Chemistry 60. Pure and Applied Chemistry 61. Record of Chemical Progress (Kresge-Hooker Scientific Library) 62. Reviews of Pure and Applied Chemistry 63. Acta Chimica Academiae Scientiarum Hungaricae 64. Acta Crystallographica 65. Advances in Physical Organic Chemistry 66. Analytical Biochemistry 67. Annali di Chimica (Rome) 68. Annual Review of Nuclear Science 69. Applied Spectroscopy 70. Archives of Biochemistry and Biophysics 71. Biochemis try 72. Canadian Chemical Processing 73. Chemical & Engineering Data Series 74. Chemisches Zentralblatt 75. Industrial and Engineering Chemistry, Product Research and Development 76. Inorganic Chemistry 77. Journal of Chromatography 78. Journal of Gas Chromatography 79* Journal of Pharmaceutical Sciences 80. Journal of Research of the National Bureau of Standards, A 81. Journal of the Electrochemical Society 82. Journal of the Indian Chemical Society 145 Table 38 (Continued)

83. Journal of the Indian Institute of Science 84. Journal of the Society of Chemical Industry (London) 85. Koninklijke Nederlandse Akademie van Wetenschappen, Proceedings, Series A 86. Memoirs of the Faculty of Science, Kyushu University, Series C. 87. Oil & Soap 88. Progress in Reaction Kinetics 89. Ricerca Scientifica 90. Spectrochimica Acta 91. Steroids 92. Svensk Kemisk Tidskrift 93. Yakugaku Zasshi 94. Zeitschrift fuer Physikalische Chemie (Leipzig) 95. Zeitschrift fuer Physiologische Chemie

journal title actually cited; changes to a later form of title

have been made only in the very few cases where citations

were made to both an earlier and a later title.

Table 39 lists the journals by reference number in

rank order (most cited to least cited) for the total unique

citations and total gross citations, giving the number of

citations, the corresponding percentage of the total, the

cumulated number of citations, and the corresponding percentage of the total. Tables 40, 41, and 42 give similar data for the

three sections, T/l, R/D, and Exp respectively.

These tables show the typical journal distribution pattern found in citation studies, with a large number of citations being made to a small number of journals and a Table 39 Journal Distribution for Total

Unique Cltfldons Grot. clc«tlon» Journal * Cutmilatct f Journal « Cumulated 1 M"H No. No 1 1 412 38 oo 412 38 00 1 770 39 35 770 39 35 2 2 100 9 93 512 4723 9 198 lo 12 968 49 46 3 3 97 8 95 609 56 18 3 146 7 46 1114 56 92 t 4 46 4 24 655 60 42 4 63 3 22 1177 60 14 5 5 35 3 23 690 63 65 5 49 2 50 1226 62 65 6 6 22 2 03 712 65 68 6 42 2 15 1268 64 79 7 7 20 1 85 73? 67 53 20 42 1310 66 94 8 a IB 1 66 750 69 19 10 35 179 1345 68 73 9 9 16 1 48 766 70 66 8 32 1 64 1377 70 36 10 10 16 1 48 782 72 14 7 29 1 48 1406 71 64 11 11 15 1 38 797 73 52 21 29 1435 73 33 12 12 13 1 20 810 74 72 9 28 1 43 1463 74 76 13 13 12 1 11 822 75 83 11 27 1 38 1490 76 14 14 14 12 1 11 834 7694 12 24 123 1515 77 36 15 15 12 1 11 646 78 04 15 24 1538 78 59 16 16 12 1 11 858 79 15 14 23 1 18 1561 79 76 IT 17 11 1 01 869 80 17 23 23 1584 60 94 18 18 10 092 879 81 09 16 IS 082 1600 81 76 19 19 10 889 82 01 17 IS 1616 82 58 SO 20 10 899 82 93 13 15 0 77 1631 83 34 21 21 9 0 83 908 83 76 19 15 1646 84 11 9? 22 8 0 74 916 B4 50 43 15 1661 B4 87 S3 23 8 924 85 24 38 13 0 66 1674 B5 54 s* 24 7 0 65 931 85 89 42 13 16B7 B6 20 85 25 7 938 86 53 25 11 0 56 1698 86 77 26 26 7 945 87 18 26 11 1709 87 33 27 27 6 0 55 951 87 73 68 11 1720 87 89 28 28 6 957 86 28 18 10 0 51 1730 88 40 29 29 5 0 46 962 68 75 2S 10 1740 88 91 30 30 4 0 37 966 89 11 24 10 1750 89 42 31 31 4 970 89 48 37 10 I76O 89 93 32 32 4 974 89 85 44 10 1770 90 44 33 33 4 978 90 22 28 9 0 45 1779 90 90 34 34 4 982 90 59 55 9 17BB 91 36 35 35 4 986 90 96 27 7 0 36 1795 91 72 36 36 4 990 91 33 29 7 1802 92 OB 37 37 3 993 91 61 31 7 1809 92 44 38 38 3 996 91 88 88 7 1816 9? 80 39 39 3 999 92 16 33 6 0 31 1822 93 10 40 40 3 1002 92 44 32 5 0 26 1827 93 36 41 41 3 1005 92 71 34 5 1832 93 61 42 42 3 1008 92 99 35 •> 1817 91 87 13 43 3 loll 93 27 36 5 1842 94 12 44 44 3 1014 93 5' 70 5 1B47 94 3B 45 45 3 1017 93 82 30 4 0 SO 1851 94 58 46 46 2 0 18 1019 94 00 40 4 0 20 1855 94 79 47 47 2 1021 94 19 41 4 1859 94 99 43 48 2 1023 94 37 45 4 1863 95 20 49 49 2 1025 94 50 49 4 IB67 95 40 50 50 S 1027 94 74 50 4 1B71 95 61 51 51 2 1029 94 93 56 4 1875 95 81 52 52 2 1031 95 11 60 4 1879 96 01 53 53 2 1033 95 30 67 4 1883 96 22 54 54 2 1035 95 48 79 4 1887 96 4? 55 55 2 1037 95 66 86 4 1891 96 63 56 56 2 1039 95 85 93 4, 1895 96 83 57 57 2 1041 96 03 39 3 0 15 1898 9699 58 58 2 1043 96 22 47 3 1901 97 14 59 59 2 1045 96 40 48 3 1904 97 27 60 60 2 1047 96 59 59 3 1907 97 45 61 61 2 1049 96 77 85 3 1910 97 60 62 62 2 1051 96 96 46 2 0 10 1912 97 70 £3 £3 0 09 1052 97 05 51 2 1914 97 80 64 64 1053 97 14 52 2 1916 97 90 65 65 1054 97 23 53 2 1918 9B 01 66 66 1055 97 32 54 2 1920 93 11 67 67 1056 97 42 57 2 1922 9B21 68 68 1057 97 51 58 1924 98 31 69 69 1056 97 60 61 1926 9B 42 70 70 1059 97 69 62 1928 9B 52 71 71 1O60 97 79 65 1930 98 62 72 72 1061 97 88 75 1932 98 72 73 73 1062 97 97 81 1934 93 82 74 74 1063 98.06 91 1936 98 93 75 75 1064 98 15 63 0 05 1937 98 98 76 76 IO65 9B25 64 1938 99 03 77 77 1066 98.34 66 1939 99 08 78 78 I067 98 43 69 1940 99 13 79 79 lose 98.52 71 1941 99 18 80 80 1069 98 62 72 1942 99 23 81 81 1070 98 71 73 1943 99 28 82 8? 1071 98.80 7 1944 99 34 83 83 1072 98 89 76i 1945 99 39 84 84 1073 98 99 77 1946 99 44 65 85 1074 99 08 78 1947 99 49 86 86 1075 99 17 80 1948 99 54 87 87 1076 99 26 82 1949 99 59 88 88 1077 99 35 83 1950 99 64 89 89 1078 99 45 B4 1951 99 69 90 90 1079 99 54 87 195» 99 74 91 91 1080 99.63 89 1953 99.80 9? 92 1081 99 72 90 1954 99.85 93 93 100? 99 a? 92 1955 99.90 94 94 1083 99 91 94 1956 99 95 95 95 1084 100 00 95 1957 100 00 Table 40 Journal Distribution for T/l

Unique Citations Gross Citations Journal * Cumulated

1 1 126 j6 42 126 36 42 1 174 34.66 174 34 66 2 2 29 8 38 155 44.80 2 42 8.37 216 43.03 3 3 29 184 53 18 3 37 7.37 253 50.40 4 5 21 6 07 205 59.25 5 25 4.98 278 55 38 5 4 12 3.47 217 62 72 23 20 3.98 298 59.36 6 8 10 2.89 227 65.6I 8 16 3.19 314 62 55 7 12 7 2 02 234 67 63 20 15 2 99 329 65 54 8 20 6 1 73 240 69.36 4 14 2 79 343 68.33 9 23 6 246 71 10 21 11 2.19 35"» 70 52 10 15 5 1.45 251 72.54 12 10 1.99 364 72 51 11 6 4 1 16 255 73.70 9 7 1.39 371 73.90 12 7 4 259 74 64 15 7 378 75 30 13 9 4 263 75.79 42 7 385 76.69 14 19 4 267 76 95 43 7 392 78.09 15 11 3 0 87 270 77.81 6 6 1 20 398 79 28 16 17 3 273 78.67 11 6 404 80.48 17 21 3 276 79.54 37 6 410 8I.67 18 22 3 279 80 40 7 4 0.80 414 82 47 19 24 3 282 81.50 10 4 418 83.27 20 25 3 285 82 37 14 4 422 84 06 21 42 3 288 83.24 19 4 426 84.86 22 43 3 291 84 10 24 4 430 85.66 23 10 2 0 58 293 84 68 33 4 434 86.45 24 13 2 295 85 26 44 4 438 87 25 25 16 2 297 85 84 50 4 442 88.05 26 26 2 299 86 42 17 3 0 60 445 88.65 27 30 2 301 86 99 ?? 3 448 89.24 28 31 2 303 87 57 25 3 451 89.84 29 33 2 305 88 15 68 3 454 90 44 30 36 2 307 88 73 88 3 457 91 04 31 37 2 309 89 31 13 2 0.40 459 91 43 32 38 2 311 89 88 16 2 461 91 83 33 50 2 313 90 46 26 2 463 92.23 34 54 2 315 91.04 30 2 465 92 63 35 14 0 29 316 91 33 31 2 467 93.03 36 18 317 91.62 36 2 469 93 43 37 27 318 91 91 38 2 471 93.82 38 32 319 92 20 40 2 473 94 22 39 3* 320 92.49 5"» 2 175 94.62 40 39 321 92 77 85 2 477 95 02 41 40 322 93.06 18 0 20 478 95.22 42 41 323 93 35 27 479 95.42 43 44 324 93 64 32 480 95 62 44 45 325 93 93 34 481 95 82 »5 48 326 94.22 39 482 96 02 46 52 0 29 327 94.51 41 0.20 483 96 22 47 55 328 94.80 45 484 96.41 48 56 329 95.09 48 485 96.61 49 60 330 95.38 52 485 96.81 50 63 331 95 66 55 487 97 01 51 66 33? 95 95 56 488 97.21 52 67 333 96 24 60 489 97.41 53 68 334 96.53 63 490 97.61 51 69 335 96 82 66 491 97.81 55 70 336 97.11 67 492 98.01 56 71 337 97.40 69 493 98.21 57 74 338 97.69 70 494 98.41 58 75 339 97.98 71 495 98.61 59 77 340 98.27 74 496 98.80 60 79 341 98 56 75 497 99.00 61 85 342 98.84 77 498 99.20 62 86 343 99.13 79 499 99.40 63 88 344 99 42 86 500 99.60 64 89 345 99.71 89 501 99.80 65 91 346 100.00 91 502 100.00 Table 41 Journal Distribution for R/D

Unlquo Citations Gross Cltatloni Journal * Cumulated * Journal * Cumulated % Sank No No

1 1 215 41.43 215 41 43 1 396 40 70 396 40.70 2 2 44 8.48 259 49 90 2 8B 9 04 484 49 74 3 3 44 303 58.38 3 67 6.89 551 56.63 4 6 17 3 28 320 61.66 6 28 2.88 579 59.51 5 4 12 2 31 332 63.97 20 27 2.77 60S 62.28 6 5 11 2.12 343 66 09 10 26 2 67 632 64 95 7 9 10 1.93 353 68 OS 5 21 2 16 653 67.ll 8 10 10 363 69 94 4 18 1 85 671 68.96 9 11 10 373 71 87 8 18 689 70.81 10 7 9 1 73 382 73.60 9 18 707 72 66 11 16 9 391 75 34 11 18 725 64 51 12 13 8 1 54 399 76 88 15 13 1 34 738 75.85 13 17 7 1.35 406 78 23 21 13 751 77.18 14 8 6 1 16 412 79.38 7 12 1 23 763 78.42 15 21 6 418 80 54 16 12 775 79.65 16 28 6 424 81.70 17 12 787 80.88 17 15 5 0 96 429 82 66 13 11 1 13 798 82 01 18 22 5 434 83 62 12 9 0.92 607 82 94 19 12 4 0.77 438 84 39 28 9 816 83 86 20 18 4 44a 85 16 38 8 0 82 824 84 69 21 19 4 446 85 93 43 8 832 85 51 22 20 4 450 86 71 68 8 840 86 33 23 29 4 454 87 48 14 6 0 62 846 86 95 24 14 3 0 58 457 88 05 22 6 852 87 56 25 24 3 460 88 63 42 6 858 88 18 26 25 3 463 89 21 55 6 864 88 80 27 27 3 466 89 79 24 5 0 51 869 89 31 28 32 3 469 90.37 25 5 874 89.83 29 34 3 472 90 94 44 5 879 90 34 30 23 2 0 39 474 91 33 18 4 0 41 883 90 75 31 31 2 476 91.71 19 4 887 91 16 32 33 2 478 92 10 27 4 891 91 57 33 39 2 480 92 49 29 4 895 91 98 34 40 2 48? 9S 87 32 4 899 9S 39 35 41 2 484 93.26 34 4 903 92 81 36 44 2 486 93 64 37 4" 907 93 22 37 45 2 468 94.03 49 4 911 93 63 3B 46 2 490 94 41 88 4 915 94 04 39 49 2 492 94 80 23 3 0 31 918 94 35 40 51 2 494 95.18 41 3 921 94 66 41 58 2 496 95 57 45 3 924 94.96 42 59 2 498 95.95 59 3 927 95 27 43 62 2 500 96.34 60 3 930 95 58 44 26 0 19 501 96.53 86 3 933 95 89 45 30 502 96 72 31 2 0 21 935 96 09 46 35 0 19 503 96.92 33 2 0 21 937 96 30 47 36 504 97 11 35 2 939 96 51 48 37 505 97 30 39 2 941 96.71 49 38 506 97.50 40 2 943 96 92 50 47 507 97.69 46 2 945 97.12 51 52 508 97 88 51 2 947 97 33 52 56 509 98.07 56 2 949 97 53 53 7 510 98 27 58 2 951 97 74 54 650 511 98 46 62 2 953 97 94 55 61 512 98 65 65 2 955 98 15 56 64 513 98.84 70 2 957 - 98 36 57 65 514 99 04 26 0 10 95B 98 46 58 76 515 99 23 30 959 98.56 59 80 516 99.42 36 960 98 66 6o 81 517 99 61 47 961 98.77 61 90 518 99 81 52 962 98 87 62 9S 519 100,00 57 963 98 97 63 61 964 99 08 64 64 965 99 18 65 67 966 99 28 66 75 967 99.38 67 76 968 99 49 68 80 969 99 59 69 81 970 99.69 70 85 971 99.79 71 90 972 99 90 72 92 973 100.00 Table 42

Journal Distribution for Exp

Unique Citations Gross Citations Journal 56 Cumulated % Journal

1 1 71 32.42 71 32.42 1 200 41.49 200 41.49 2 2 27 12.33 98 44.75 2 68 14.11 268 55.60 3 3 24 IO.96 122 55.71 3 42 8.71 310 64.32 4 4 22 10.05 144 65.75 4 31 6.43 341 70.75 5 14 8 3.65 152 69.41 7 13 2.70 354 73.44 6 7 7 3.20 159 72.60 14 13 367 76.14 7 18 5 2.28 164 74.89 6 8 1.66 375 77.80 8 10 4 1.83 168 76.71 26 8 383 79.46 9 26 4 172 78.54 19 7 1.45 390 80.91 10 5 3 1.37 175 79.91 10 5 1.04 395 81.95 11 35 3 178 81.28 12 5 400 82.99 12 8 2 0.91 180 82.19 18 5 405 84.02 13 9 2 I82 83.11 15 4 0.83 409 84.85 14 11 2 184 84.02 93 4 413 85.68 15 12 2 186 84.93 5 3 O.62 416 86.31 16 13 2 188 85.84 8 3 419 86.93 17 15 2 190 86.76 9 3 422 87.55 18 19 2 192 87.67 11 3 425 88.I7 19 27 2 194 88.58 25 3 428 88.80 20 53 2 196 89.50 29 3 431 89.42 21 6 0.46 197 89.95 31 3 434 90.04 22 16 198 90.41 35 3 437 90.66 23 17 199 90.87 38 3 440 91.29 24 24 200 91.32 79 3 443 91.91 25 25 201 91.78 13 2 0.41 445 92.32 26 29 202 92.24 16 2 447 92.74 27 30 203 92.69 27 2 449 93.15 28 36 204 93.15 36 2 451 93-57 29 47 205 93.61 47 2 453 93.98 30 48 206 94.06 48 2 455 94.40 31 55 207 94.52 53 2 457 94.81 32 57 208 94.98 55 2 459 95.23 33 61 209 95.43 67 2 461 85.64 3^ 72 210 95.89 70 2 463 96.06 35 73 211 96.35 17 0 21 464 96.27 36 78 212 96.80 22 465 96.47 37 82 213 97.26 24 466 96.68 38 83 214 97-72 30 467 96.89 39 84 215 98.17 44 468 97-10 40 87 216 98.63 56 469 97-30 41 93 217 99.09 57 470 97.51 42 94 218 99.54 61 471 97.72 43 95 1 219 100.00 72 472 97-93 44 73 473 98.13 45 78 474 98.34 46 81 0.21 475 98.55 47 82 476 98.76 48 83 477 98.96 49 84 478 99.17 50 87 479 99.38 51 91 480 99.59 52 94 481 99-79 53 95 482 100.00 150

corresponding small number of citations being made to a large

number of journals.

The number of citations and the number of journals cited

with the corresponding percentages, and the resulting average

number of citations per journal for each section, unique

citations and gross citations, are given in Table 43.

Table 43

Number of Journals Cited and Average Concentration of Citations/Journal by Section

Total T/I R/D Exp

unique citations 1084 346 519 219 $ 31.92$ 47.88$ 20.20$ journals cited 95 65 62 43 $ 68.42$ 65.26$ 45.26$ average unique citations/ journal cited 11.41 4.32 8.37 5-09 gross citations 1957 502 973 482 % 25.65$ 49.72$ 24.63$ journals cited 95 65 72 53 $ 68.42$ 75.79$ 55.79$ average gross citations/ journal cites 20.60 7.72 13.51 9.09

A total of 95 different journals were cited, 65 of these

in the T/l, 62 (72 with the addition of the repeat citations) 151

in the R/D, and 43 (53 with the addition of the repeat cita­

tions) in the Exp. The sum of the journals cited in each

section is greater than the total number of journals cited

since one journal can be cited in one, two or all three

sections. This is illustrated in Table 44.

Table 44

Number of Journals Cited in One, Two, or All Three Sections

Unique citations Gross Citations

three 25 33

two T/I + R/D 18 20 R/D + Exp 5 6 T/I + Exp 2 3 one T/I 20 9 R/D 14 13 Exp 11 11 total 95 95

It might be noted that the number of journals cited only in the T/I decreases with the addition of the repeat cita­ tions (20 - 9) while the number of journals cited only in the R/D and the Exp remain about the same. This again might 152

indicate that some of the repeat citations are to articles

first used in the T/I and then repeated in the other sections.

From Table 43, the order for the number of journals

cited for the unique citations is T/l>R/D>Exp while the

corresponding order for the number of citations is R/D>T/l>Exp.

The combination of the larger number of citations and the

smaller number of journals produces an average concentration

of citations/journal for the R/D that is greater than that for

the T/I. For the unique citations, the concentration of

citations per journal is greatest in the R/D (8.37)» followed

by the T/I (5.32), and the Exp (5.09) corresponding to the

order of the number of citations for each section. This is

somewhat to be expected due to the nature of the distribution

of the citations among the journals where a larger proportion

of the citations are to a small proportion of the journals.

Thus an increase in the number of citations should be greater

than the corresponding increase in the number of journals cited and a larger concentration of citations per journal would be obtained. This would apply to both the comparison of the three sections and to the comparison of unique cita­

tions and gross citations within each section.

When the repeat citations are added, the number of journals cited by the total and the T/I can not increase and 153 the order of the number of journals cited is now identical to the order for the number of citations R/D>T/l>Exp, however, the concentration of citations/journals is now in the order

R/D>Exp>T/I. This is due to the larger difference in the number of journals cited in the T/l and the Exp (65:53 respec­ tively) compared with the small difference in the corresponding number of citations (502:482 respectively) thus causing a reversal in order. This would indicate that the citations from the T/l and the Exp are not behaving in a similar manner with respect to journal distribution. The most obvious difference is that the Exp has a smaller number of journals cited. This may be related to "the fact that all journals do not present experimental data (e.g., review journals, communi­ cations, etc.) and therefore the population of journals for the Exp to draw from for citing may actually be smaller than that for the T/l.

The next step was to look at the number of journals needed to produce a given percentage of citations. In all cases, journals No. 1, 2, and 3 (JACS, JOC, and the Journal of the Chemical Society respectively) are ranked 1, 2, and 3 respectively and provide at least 50$ of the citations.

Information was extracted from Tables 39, 40, 41, and 42 and presented in Table 45 in order to compare how many cited 154 journals, and the corresponding percentage, it takes to produce 75$ and 90$ of the citations.

Table 45 Number of Cited Journals Required to Give 75$ and 90$ of Citations by Section

Total T/I R/D Exp journals cited 95 65 62 43 journals required to give 75$ unique citations 13 13 11 8 $ of journals cited 13.68$ 20.00$ 17.74$ 18.60$ 90$ unique citations 33 33 28 22 $ of journals cited 34.74$ 50.77$ 45.16$ 51.16$ journals cited 95 65 72 53 journals required to give 75$ gross citations 13 12 12 6 $ of journals cited 13.68$ 18.46$ 16.67$ 11.32$ 90$ gross citations 32 29 29 21 $ of journals cited 33.68$ 44.62$ 40.28$ 39.62$

In,each case, the addition of the repeat citations lowered the percentage of journals needed to achieve the given per­ centage of citations, indicating that the repeat citations are primarily to journals already heavily cited. In the T/l, since no new journals can be cited, the decrease in the percentage of journals needed is due only to the decrease 155 in the number of journals needed. For the R/D and the Exp., the decrease is primarily due to the increase in the total number of journals cited.

Data have also been extracted from the other end of

Tables 39, 40, 41, and 42. The number of journals having only one citation, the corresponding percentage, the cor­ responding percentage of the total citations, and the ratio of these percentages are presented in Table 46.

Table 46

Number of Journals Cited Only Once by Section

Total T/I R/D Exp journals, 1 citation 33 31 19 23 journals cited 95 65 62 43 $ journals, 1 citation 34.74$ 47.69$ 30.65$ 53.49$ unique citations )84 346 519 219 $ to journals, 1 citation 3.04$ 8.96$ 3.66$ 10.50$ $ unique citations/ $ journals 8.75 18.79 11.94 19.63 journals 1 citation 21 25 ~~16,. 19 journals cited 95 65 72 53 $ journals, 1 citation 22.11$ 38.46$ 22 .22$ 35.85$ gross citations 57 502 973 482 $ to journals, 1 citation 1.071.07$$ 4.98$ 1.64$ 3.94$ $ gross citations/ $ journals 4.84 12.95 7.38 10.99 156

The number of journals with only one citation, the per­ centages, and the ratio of the percentages are smallest for the R/D. About 50$ of the unique citations and about 35$ of the gross citations in the T/I and the Exp are to journals with only one citation, while the percentage for the R/D is considerably lower in each section. This is most likely a consequence of the larger number of citations in the R/D; the number of journals cited does not increase proportionately with the number of citations. As would be expected, the addition of the repeat citations decreases the percentage of journals with only one citation.

The data from Tables 43, 45, and 46 also indicate that the values obtained for the total in each case do not approximate either the behavior of any one section or an average of the three sections, again because one title can be cited in any or all of the sections. Various models have been proposed to describe the exponential journal distributions obtained in Tables 39, 40, 41, and 42. Since the primary purpose of this study was to compare the journal distributions by section, the comparison rather than the specific model becomes the important factor. It was decided to use the model C = aT where C = the number of citations,

T = journal (title) rank order, and a and b are constants. 157 By taking logarithms of both sides, the equation becomes

log C = log a + b log T which is the equation for a straight line where log a = constant (C intercept) and b = constant

(slope). The data from Tables 39, 40, 41, and 42 were there­ fore plotted in log-log form in Figures 1, 2, 3, and 4 and a computer program was used to perform a least squares calcu­ lation in order to obtain the best-fitting straight line for the data. This program calculated the constants a and b and an error factor according to the equations given in Table

47. The standard error of estimate, S for the y deviations

(in this case the C deviations) about a fitted curve is obtained by taking the square root of the error factor.

The results are given in Table 48.

these calculated best-fitting straight lines have been added to Figures 1, 2, 3, and 4 along with the corresponding equation and the standard error of estimate. For both the unique citations and the gross citation, the slope (b) of the calculated straight line is larger (negatively) for the

R/D than for the Exp or the T/I; however, the total has the largest (negative value). In each case, the slope (b) increases (negatively) with the addition of the repeat cita­ tions, again indicating that the repeat citations are primarily to journals already heavily cited, that the increase in 158 IOJO-T

100-r

Figure 1. Journal Distribution (Log-Log Form) of Total Unique (Lower Curve) and Total Gross (Upper Curve) Citations

100-

100

Figure 2. Journal Distribution (Log-Log Form) of T/l Unique (Lower Curve) and T/l Gross (Upper Curve) Citations 159

100 ••

IO ••

Figure 3« Journal Distribution (Log-Log Form) of R/D Unique (Lower Curve) and R/D Gross (Upper Curve) Citations

100

io -•

100

Figure 4. Journal Distribution (Log-Log Form) of Exp Unique (Lower Curve) and Exp Gross (Upper Curve) Citations Table 47

Least Squares Calculation

N N N N 0 N 9 b = (N S x.y, - 2 x 2 y.)/(N 2 x * - (2 x,)*) i=l x x i=l 1 i=l 1 i=l 1 i=l 1

N N N a = ( 2 x.y. - b 2 x. )/( 2 x,) i=l x x i=l x i=l 1

N • „ error = 2 (a + bx. - y.) /N i=l x 1

N S = 2 d 2 /N = Vferror y i=l 1 1

where N = number of points plotted x. = value of the x coordinate for point i (in case T.) y. = value of the y coordinate for point i (in case C. or CC.) d. = difference between calculated and observed values of y. 161 Table 48

Least Squares Calculation for Journal Distribution by Section

Total T/I R/D Exp unique citations

b - 1.329 - 1.096 - 1.211 - 1.183 log a 2.576 1.862 2.144 1.762 a 376.7 72.78 139.3 57.81 error ' 0.00774 0.00697 0.00750 0.01198 S 0.088 0.083 0.087 0.109 y gross citations b - 1.458 - 1.259 - 1.331 - 1.297 log a 2.985 2.216 2.547 2.142 a 966.1 164.4 352.4 138.7 error 0.01852 O.OO652 0.01212 0.00729 S O.I36 0.081 0.110 O.O85 v

journals cited is not proportionate to the increase in the number of citations. From the error factor, it would appear that the model fits the behavior of the T/l the best. The addition of the repeat citations improves the fit of the model for the

Exp but worsens the fit of the model for the R/D and the total.

In these latter two cases, the data fall below the calculated straight line in the first part of the curve (heavily cited journals) and then rise above the calculated straight line in the last part of the curve. The improvement in the fit of 162 the model for the Exp with the addition of the repeat cita­ tions appears to be due to a straightening out of the first part of the curve, an increase in the number of citations to the already heavily cited journals. This is again the result of the relative position of the Exp within the source article.

Another method of presenting the data from Tables 39,

40, 41, and 42 is to plot the cumulated citations against the journals (in rank order) in semi-log form as in Figures 5, 6,

7, and 8. The model used to describe cumulated journal distribution was CC = a + b log T where CC = cumulated cita­ tions, T = journal (title) rank order, a = constant (C intercept) and b = constant (slope). The same computer program was again used to perform a least squares calculation and the results are given in Table 49.

The resulting best-fitting straight lines, the cor­ responding equation, and the standard error of estimate have been added to Figures 5, 6, 7, and 8. Again the slope

(b) is the largest for the total, followed by the R/D, T.I and the Exp for both unique citations and gross citations, with an increase in the slope (b) in each case with the addition of the repeats. This may indicate that the sample size is the important factor in the determination of the slope (b) rather than difference in journal distribution in 163

600-

200-1 • I 10 100

Figure 5. Cumulated Journal Distribution (Semi-Log Form) of Total Unique (Lower Curve) and Total Gross (Upper Curve) Citations

100

Figure 6. Cumulated Journal Distribution (Semi-Log Form) of T/l Unique (Lower Curve) and T/l Gross (Upper Curve) Citations I 164

1000

aoo

100

Figure "J. Cumulated Journal Distribution (Semi-Log Form) of R/D Unique (Lower Curve) and R/D Gross (Upper Curve) Citations

100

Figure 8. Cumulated Journal Distribution (Semi-Log Form)"of Exp Unique (Lower Curve) and Exp Gross (Upper Curve) Citations 165

Table 49

Least Squares Calculation for Cumulated Journal Distribution by Section

1 Total T/I R/D Exp unique citations b 327-206 119 .294 174.122 82.011 a 461.595 129.786 212.803 87.216 error 239.951 4.139 13.086 23.949 S 15.49 2.04 3.62 4.89 y gross citations b 596.009 185,687 322.692 144.043 a 835.932 175.183 394.701 240.591 error 874.820 36,744 105.929 98.343 S 29.58 6.06 10.29 9.92 y

the three sections. The fit of the model is the best in the case of the T/l and the worst for the total. In each case, the fit for the unique citations is better than that for the gross citations primarily due to the lowering of the end of the curve below that of the calculated straight line, which is again related to the sample size.

Another very important factor in the journal distribution by section is the ranking order for the journals, the order in which the journals are arranged for each section when ranked from the most cited to the least cited. The journal 166 rank represents different journals for each section since the journals are arranged separately in rank order for each section. For example, for the total unique citations, the journal ranked 10 is journal No. 10 while the same rank of 10 represents journal No. 15 for the T/l, journal No. 7 for the

R/D, and journal No. 5 for the Exp. To illustrate the fact that the rank does not always indicate the same title in each section, the information from Tables 40, 41, and 42 have been reversed and presented in Table 50 (unique citations) and Table 51 (gross citations) so that the rank for each journal in each section is given.

The ranks of journals No. 1-12 for the unique citations are repeated in Table 52. These rankings were then reduced to a 1-12 ranking and a Kendall coefficient of concordance

W test was performed to test the null hypothesis that the rankings of journals No. 1-12 by section was completely random.

A value of W = 0.759 producing a Chi Square = 25.05 was obtained. This is significant at the 0.05 level and therefore the null hypothesis can be rejected and it can be concluded that the journal rankings by section are significantly similar.

Inspection shows that the greatest agreement comes with journals No. 1-3 which are ranked 1-3 in each and every section. These three journals were therefore removed and 167

Table 50

Ranking of Cited Journals, Unique Citations, by Section

T/I R/D Exp T/I R/D Exp T/l R/D Exp Journal Journal Journal No. Rank No. Rank No. Rank

1 1 1 1 31 28 31 61 55 33 2 2 2 2 32 38 28 62 43 3 3 3 3 33 29 32 63 50 4 5 5 4 34 39 29 64 56 5 4 6 10 35 46 11 65 57 6 11 4 21 36 30 47 28 66 51 7 12 10 6 37 31 48 67 52 8 6 14 12 38 32 49 68 53 9 13 7 13 39 40 33 69 54 10 23 8 8 40 41 34 70 55 11 15 9 14 41 42 35 71 56 12 7 19 15 42 21 72 34 13 24 12 16 43 22 73 35 14 35 24 5 44 43 36 74 57 15 10 17 17 45 44 37 75 58 16 25 11 22 46 38 76 58 17 16 13 23 47 50 29 77 59 18 36 20 7 48 45 30 78 36 19 14 21 18 49 39 79 6o 20 8 22 50 33 80 59 21 17 15 51 40 81 60 22 18 18 52 46 51 82 37 23 9 30 53 20 83 38 24 19 25 24 54 34 84 39 25 20 26 25 55 47 31 85 61 26 26 44 9 56 48 52 86 62 27 37 27 19 57 53 32 87 40 28 16 58 41 88 63 29 23 26 59 42 89 64 30 27 45 27 60 49 54 90 61 91 65 92 62 93 41 94 42 95 43 168 Table 51

Ranking of Cited Journals, Gross Citations, by Section

T/I R/D Exp T/I R/D Exp T/l R/D Exp Journal Journal Journal No. Rank No. Rank No. Rank

1 1 1 1 31 35 45 21 61 63 42 2 2 2 2 32 43 34 62 54 3 3 3 3 33 23 46 63 53 4 8 8 4 34 44 35 64 64 5 4 7 15 35 47 22 65 55 6 15 4 7 36 36 59 28 66 54 7 18 14 5 37 17 36 67 55 65 33 8 6 9 16 38 37 20 23 68 29 22 9 11 10 17 39 45 48 69 56 10 19 6 10 40 38 49 70 57 56 34 11 16 11 18 41 46 40 71 58 12 10 18 11 42 13 25 72 43 13 31 17 25 43 14 21 73 44 14 20 23 6 44 24 29 39 74 59 15 12 12 13 45 47 41 75 60 66 16 32 15 26 46 50 76 67 17 26 16 35 47 60 29 77 61 18 41 30 12 48 48 30 78 45 19 21 31 9 49 37 79 62 24 20 7 5 50 25 80 68 21 9 13 51 51 81 69 46 22 27 24 36 52 49 61 82 47 23 5 39 53 31 83 48 24 22 27 37 54 39 84 49 25 28 28 19 55 50 26 32 85 40 70 26 33 57 8 56 51 52 40 86 63 44 27 42 32 27 57 62 41 87 50 28 19 58 53 88 30 38 29 33 20 59 42 89 64 30 34 58 38 60 52 43 90 71 91 65 51 92 72 93 14 94 52 95 53 Table 52

Kendall Coefficient of Concordance W for Journals No. 1-12, Unique Citations

Journal No. 1 2 3 4 5 6 7 8 9 10 11 12 Rank

T/I 1 2 3 5 4 11 12 6 13 23 15 7 R/D 1 2 3 5 6 4 10 14 7 8 9 19 Exp 1 2 3 4 10 21 6 12 13 8 14 15 ranked 1-12 T/I 1 2 3 5- 4 8 9 6 10 12 11 R/D 1 2 3 5 6 4 10 11 7 8 9 •J Exp 1 2 3 4 7 12 5 8 9 6 10 11 sum of ranks 3 6 9 14 17 24 24, 25 26 26 30 30 mean = 19-5 s = 977 w = 0.75S) Chi Square == 25.05 critical value (11 degrees3 of freedom , o. 05 level of signifi- cance) = 19 .68

Journals No. 4 -12. T/I 2 1 5 6 3 7 9 8 4 R/D 2 3 1 7 8 4 5 6 9 Exp 1 4 9 2 5 6 3 7 8 sum of ranksi 5 8 15 15 16 17 17 21 21 mean = 15 s = 230 w = 0.426 Chi Square =•• 10.22 critical value (8 degrees of freedom, 0.05 level of signifi­ cance) = I5.5I. 170

the same Kendall coefficient of concordance W test was per­

formed on the remaining eight journals, journals No. 4-12. A value of W - 0.426 producing a Chi Square = 10.22 was obtained.

This is not significant at the 0.05 level, thus the null hypothesis can not be rejected and the ranking order of journals No. 4-12 by section can be accepted as possibly arising solely from sampling variation.

The data for journals No. 1-12 for the three sections, gross citations, were treated in a similar manner in Table 53*

Again, the null hypothesis can be rejected at the 0.05 level for journals No. 1-12 but not for journals No. 4-12. However, the null hypothesis can not be rejected for journals No. 1-12 at the 0.01 level of significance (critical value, 11 degrees of freedom = 24.72).

Since the ranking for the total gross citations is not the same as for the total unique citations, it was decided to look at these first 12 ranked journals from Table 39 as well: No. 1, 2, 3, 4, 5, 20, 10, 8, 7, 21, and 9. However, on investigation, it was found that journals No. 20 and No. 21 were not cited at all by the Exp and so were replaced by the next ranked journals, No. 11 and No. 12. This results in journals No. 1-12 in a slightly different order and since W does not depend on the order of the entities to be ranked, the same values for W and Chi Square would be obtained. 171 Table 53 Kendall Coefficient of Concordance W for Journals No. 1-12, Gross Citations

Journal No. 1 2 3 4 5 6 7 8 9 16 11 12 rank T/I 1 2 3 8 4 15 18 6 11 19 16 10 R/D 1 2 3 8 7 4 14 9 10 6 11 18 Exp 1 2 3 4 15 7 5 16 17 10 18 11 ranked 1-12 T/I 1 2 3 6 4 9 11 5 8 12 10 7 R/D 1 2 3 7 6 4 11 8 9 5 10 12 Exp 1 2 3 4 9 6 5 10 11 7 12 8 sum of ranks 3 6 9 17 19 19 27 23 28 24 32 27 mean = 19.5 s = 945 w = 0.734 Chi Square = 24.22 critical value (11 degrees of freedom, 0.05 level of signifi­ cance) = 19.68 journals No. 4-12 rank T/I 3 1 6 8 2 5 9 7 4 R/D 4 3 1 8 5 6 2 7 9 Exp 1 6 3 2 7 8 4 9 5 « sum of ranks 8 10 10 18 14 19 15 23 18 mean = 15 s = 198 w = O.367 Chi Square =8.81 1 critical value (8 degrees of freedom, 0.05 level of signifi­ cance) = 15.51 /

172

In addition, Spearman rank correlation coefficients r

were determined for the total citations and for each section

to test the null hypothesis that the rankings for journals

No. 1-12 and No. 4-12 are completely random (i.e., do not

change significantly) with the addition of the repeat cita­

tions. Since journals No. 1-12 are not always the same as

the journals ranked 1-12 in each section, the journals ranked

1-12 by unique citations in the total and for each section

were also used. The results are given in Table 54.

Table 54

Spearman Rank Correlation Coefficients for Journals No. 1-12 and Journals Ranked 1-12, Unique Citations and Gross Citations, by Section

Total T/l R/D Exp

journals No. 1-12 O.95I 0.979 O.902 O.783 journals ranked 1-12 by unique citations O.95I O.895 O.95I 0.979 critical value (N = 12, 0.05 level of significance) = O.506

journals No. 4-12 0.883 0.950 O.767 0.483 journals ranked 4-12 by unique citations O.883 0.750 O.883 0.950 critical value (N = 9, 0.05 level of significance) = 0.600 173 At the 0.05 level of significance, all r except for the S Exp journals No. 4-12, are significant and therefore the null hypothesis can be rejected and the rankings considered sig­ nificantly the same for the unique citations and the gross citations. Again, this would indicate that the repeat cita­ tions are from journals already heavily cited by unique cita­ tions.

Time Distribution

The time distribution of the citations was also investi­ gated as a property of citations in order to determine if any difference existed among the time distributions for the three sections and/or if the addition of the repeat citations produced any changes. The number of ctiations per year, along with the percentage of the total number of citations, the cumulated number of citations (in reverse chronological order), and the corresponding percentage for the total, T/l, R/D, and Exp (unique citations and gross citations) are given in

Tables 55, 56, 57, and 58 respectively.

The number of years (back in time from I966) required to produce 50$, 75$ and 90$ of the citations for the total and for each section was then determined. These data are presented in Table 59 * for comparison. Table 55 Time Distribution for Total

Unlqut Citations Crofts Citation. Year ? Cumulated * * Cumulated *

I966 33 3 04 33 3 04 79 4 04 79 4 04 1965 142 13 01 175 16 14 335 17 12 414 21 15 196* 129 11 90 304 28 04 219 11 19 633 32 35 1963 121 11 16 425 39 21 246 1? 57 879 44 92 196: 69 6 37 494 45 57 121 6 IB 1000 51 10 1961 67 6 18 561 51 75 125 6 39 1125 57 49 I960 46 4 2* 607 56 00 78 3 99 1203 61 47 1959 44 4 06 651 60 06 68 3 47 1271 64 95 195B 41 3 78 692 63 84 72 3 68 1343 6B 63 1957 39 3 60 731 67 44 65 4 34 1428 72 97 1956 30 ? 77 761 70 20 53 2 71 1481 75 68 1955 31 2 86 792 73 06 57 2 91 1538 78 59 195* 28 2 58 8?0 75 65 32 1 64 1570 80 22 1953 10 0 92 830 76 57 16 0 6? 1586 81 04 195? 2* i ?1 854 78 78 26 1 33 161? 8? 37 1951 26 2 40 880 81 18 32 1 64 1644 84 01 1950 17 1 57 897 82 75 32 1 64 I676 85 64 19»9 19 1 75 916 84 50 24 1 23 1700 86 67 19*8 12 1 11 928 85 61 26 I 33 1726 88 ?0 19*7 10 0 92 938 86 53 13 0 66 1739 88 86 19*6 5 0 46 943 86 99 7 0 36 1746 69 22 19*5 1 0 37 947 87 36 5 0 ?6 1751 89 47 19** 4 0 37 951 87 73 6 0 31 1757 69 78 1913 6 0 55 957 88 ?8 11 0 56 1768 90 34 191? 6 0 55 963 88 84 7 0 36 1775 90 70 1941 8 0 74 971 89 58 12 0 61 1767 91 31 1940 7 0 65 978 90 ?2 10 0 51 1797 91 82 1939 6 0 55 984 90 77 14 0 76 1811 92 54 1938 4 0 37 988 91 14 4 0 ?0 1815 92 74 1937 5 0 46 993 91 61 12 0 61 1627 93 36 1936 3 0 28 996 91 88 3 0 15 I830 93 51 1935 7 0 65 1003 92 53 10 0 51 1B40 94 02 1934 1 0 09 1004 9? 62 2 0 10 1S4? 94 12 1933 6 0 55 1010 93 17 7 0 36 1849 94 48 1932 6 0 55 1016 93 73 9 0 46 I658 94 94 1931 3 0 28 1019 94 00 5 0 26 IB63 95 20 1930 4 0 37 1023 94 37 4 0 20 I8S7 95 40 1929 2 0 IB 1025 94 56 2 0 10 1869 95 50 1928 2 0 18 1027 94 74 2 0 10 1871 95 61 19?7 3 0 28 1030 95 02 3 0 15 187* 95 76 1926 7 0 65 1037 95 66 11 0 56 18B5 96 32 1925 1037 95 66 1885 96 3? 1924 2« 0 . IB 1039 95 85 4. 0 . 20 1889 96 53 1923 1 0 09 1040 95 94 1 0 05 1890 96 58 1922 1 0 09 1041 96 03 1 0 05 I89I 96 63 1921 1 0 09 1042 96 13 1 0 05 189? 96 6B 1920 3 0 28 1045 96 40 4 0 10 1896 96 88 1919 3 0 28 1048 96 68 8 0 41 1904 97 27 1918 1917 1916 1 0 09 1049 96 77 5 0 ?6 1909 97 55 19»5 1911 2 0 18 1051 96 96 6 0 31 1915 97 85 1913 1912 1911 2 0 18 1053 97 14 1 0 10 1917 97 96 1910 1909 1908 1<"7 2 0.18 1055 97 32 * 0 10 1919 98 06 ' J6 3 0 28 1058 97 60 4 0 20 1923 98 26 1905 1 0 09 1059 97 69 I 0 05 1924 98 31 1901 1 0 09 1060 97 T9 1 0 05 1925 9B 36 1903 4 0 37 1064 98 15 4 0 20 1929 98 57 1902 1901 1 0 09 1065 98 ?5 2 0 10 1931 98 67 1900 1 0 09 1066 98 34 1 0 05 193? 98 72 1899 1 0 09 IO67 98 43 0 10 1934 98 B? 1898 1 0 09 1068 98 52 ;1 0 05 1935 98 86 1897 1 0,09 IO69 98 62 1 0 05 1936 98 93 1896 1895 1 0 09 1070 98 71 1 0 05 1937 98 99 1691 1893 2 0 18 1072 98 89 2 0 10 1939 99 08 169? ? 0 IB 1074 99 08 3 0 15 1943 99 23 1891 1 0 09 1075 99 17 1 0 05 1943 99 28 1890 1 0 09 1076 99 ?6 1 0 05 1944 99 34 1889 1683 I887 1 0 09 1077 99 35 4 0 20 1948 99 54 1886 1 0 09 1078 99 45 1 0 05 1949 99 59 1885 1 0.09 1079 99 54 2 0 10 1951 99 69 1801. I883 188? 1 0 09 1060 99 63 1 0,05 1952 99 74 1881 ? 0 18 108? 99 8? ? 0 10 1954 99 85 1860 1879 1878 1 0 09 1083 99 91 1 0 05 1955 99 90

1655 1 0 09 1084 100 00 ? 0 10 1957 100.00 Table 56 Time Distribution for T/l

Unique CI cations Gross Citations Year " Cumulated " Cumulated " 1966 16 4 62 16 4 62 3? 6 37 3? 6 37 1965 51 14 74 67 19 36 64 16 73 116 23 11 1964 54 15 61 121 34.97 73 14 54 189 37 65 1963 40 14 45 171 49 4? 64 16 73 273 54 38 196? 27 7 BO 198 57 23 37 7 37 310 61 75 1961 21 6 07 219 63 29 35 6 97 345 68 73 i960 14 4 05 233 67 34 16 3 19 361 7191 1959 14 4 05 247 71 39 19 3 78 380 75 70 1958 9 2 60 256 73 99 9 1 79 389 77 49 1957 12 3 47 26B 77 46 14 2 79 403 80 28 195< 6 2 31 276 79 77 15 2 99 4 IB 83 ?7 1955 11 3 18 287 6? 95 15 2 99 433 66 25 1954 12 3 47 299 86 42 13 ? 59 446 88 64 1953 1 0 29 300 86 70 1 0 20 447 89 04 1952 5 I 45 305 68 15 5 1 00 45? 90 04 1951 4 1 16 309 89 31 4 0 80 456 90 84 1950 3 0 87 312 90 17 3 0 60 459 91 43 1949 2 0 58 314 90 75 2 0 40 461 91 B3 1948 2 0 56 316 91 33 2 0 40 463 92 ?3 1947 1946 1945 1944 1943 2 0 58 318 91 91 2 0 40 465 9? 63 1942 2 0 58 320 92 49 2 0 40 467 93 03 1941 1940 1939 2 0 58 322 93 OS 2 0 40 469 93 43 1938 1937 2 0 56 324 93 64 3 0 60 472 94 02 1936 1935 2 0 58 326 94 22 2 0 40 474 94 4? 1934 1933 193? 2 0 56 328 94 80 2 0 40 476 94 Be 1931 1930 2 0 58 330 95 38 2 0 40 478 95 H 1929 1928 1927 1926 2 0 58 332 95 95 3 0 60 481 95 82 1925 1924 1923 1922 1921 1920 1919 1 0 !9 333 96 24 3 0 60 484 96 41 1918 1917 1916 1 0 29 334 96 53 3 0 60 487 97 01 1915 1914 1 0 29 335 96 82 3 0 60 490 97 61 1913 1912 1911 1910 1909 1908 1907 2 0 58 337 97 40 2 0 40 492 98 01 1906 0 29 33B 97 69 0 20 493 98 21 1905 1904 1903 0 87 341 96 55 0 60 496 98 60 1902 1901 1900 1899 0 29 342 98 84 0 40 .498 99 20 1898 1897 0 29 343 99 13 0 20 499 99 40 1S9S 1895 1894 1893 0 29 344 99 4? 0 20 500 99 60 1B92 0 29 345 99 71 0 20 501 99 60 I891 1890 1889 1888 1887 1886 I885 1884 1683 168? 1 0 29 346 100 00 1 0 20 502 loo 00 1681 I860 1679 167B Table 57 Time Distribution for R/D

Unique Citation s Gross Citations Year " Cumulated ' Cumulated

1966 15 f 89 15 2 89 42 4 3? 42 4 3? 1965 79 15 f? 94 IB 11 203 ?0 86 245 25 IB 1964 68 13 10 162 31 ?1 124 12 74 369 37 92 1963 56 10 79 218 42 00 120 12 33 489 50 26 1962 39 7 51 257 49 52 70 7 19 559 57 45 1961 32 6 17 289 55 68 63 6 47 622 63 93 i960 28 5 39 317 61 08 40 4 U 662 68 04 1959 18 3 47 335 64 55 30 3 08 692 71 12 1958 22 4 24 357 6B79 37 3 80 729 74 92 1957 16 3 08 373 71 67 36 3 70 765 78 6? 1956 17 3 28 390 75 14 27 2 77 792 81 40 1955 13 2 50 403 77 65 ?3 2 36 815 83 76 1954 7 1 35 410 79 00 6 0 8? 823 84 58 1953 4 0 71 414 79 77 5 0 51 828 85 10 195? 10 1 93 424 81 70 11 1 13 839 86 23 1951 15 2 89 439 84 59 18 I 65 657 88 06 1950 B 1 54 447 66 13 16 1 64 873 89 72 1949 9 1 73 456 87 86 11 1 13 864 90 85 1948 8 1 54 464 89 40 13 1 34 897 9? 19 1947 4 0 77 46B 90 17 4 0 41 901 92 60 1946 1 0 19 469 90 37 2 0 21 903 9? 81 1945 3 0 58 472 90 94 3 0 31 906 93 11 1944 3 0 58 475 91 52 3 O 31 909 93 42 1943 2 0 39 477 91 90 3 0 31 912 93 73 1942 1941 4 0 77 481 92 6B 4 0 41 917 94 24 1940 2 0 39 483 93 06 3 0 31 920 94 55 1939 2 0 39 485 93 45 2 0 21 922 94 76 1938 3 0 58 4B8 94 03 3 0 31 925 95 07 1937 2 0 39 490 94 41 4 0 41 929 95 48 1936 3 0 58 493 94 99 3 0 31 93? 95 79 1935 3 0 58 496 95 57 4 0 41 936 96 20 1934 1933 3 0 58 499 96 15 3 0 31 939 96 51 1932 3 0 58 50? 96 7? 5 0 51 944 97 0? 1931 1 0 19 503 96 9? 1 0 10 945 97 12 1930 1 0 19 504 97 11 1 0 10 946 97 23 1929 1928 1 0 19 505 97 30 1 0 10 947 97 33 1927 1 0 19 506 97 50 1 0 10 948 97 43 1926 1 0 19 507 97 69 3 0 31 951 97 74 1925 19?4 1 0 19 508 97 86 1 0 10 952 97 64 1923 1922 1921 1 0 19 509 98 07 1 0 10 953 97 94 1920 1 0 19 510 98 27 2 0 21 955 98 15 1919 1 0 19 511 98 46 4 0 41 959 98 56 1918 1917 1916 1 0 21 96I 98 77 1915 1914 2 0 21 963 98 97 1913 1912 1911 2 0 39 513 98 64 2 0 21 965 99 18 1910 1909 1908 1907 1906 2 039 515 99 23 2 0 21 967 99 38 1905 1904 1903 1902 1901 1 0 19 516 99 42 1 0 10 968 99 49 1900 1699 1898 1697 IB96 1895 IB94 1893 1692 1 0 19 517 99 61 1 0 10 969 99 59 1891 I890 1 0 19 518 99 61 1 0 10 970 99 69 1889 1668 1887 1 0 19 519 100 00 3 0 31 973 loo 00 1686 1885 1884 1683 188? 1861 1680 1679 I87B Table 58 Time Distribution for Exp

Unique Citations crois Cles tlons Y.ar * Cumulated " Cumulated * 1966 2 0 91 2 0 91 5 1 04 5 1 04 1965 12 5 46 14 6 39 48 9 96 53 11 00 1964 7 3 20 21 9 59 22 4 56 75 15 56 1963 15 6 65 36 16 44 42 6 71 117 24 27 1962 3 1 37 39 17 81 14 2 90 131 27 18 1961 12 6 39 53 24 ?0 27 5 60 158 32 7B I960 4 1 83 57 26 03 22 4 56 180 37 34 1959 1? 5 48 69 31 51 19 3 94 199 41 29 1958 10 4 57 79 36 07 26 5 39 225 46 68 1957 11 502 90 41 10 35 7 26 ;6o 53 94 1956 5 2 28 95 43 38 11 2 28 271 56 2? 1955 7 3 20 102 46 58 19 3 94 !90 60 17 1954 9 4 11 111 50 68 11 2 ?3 301 62 45 1953 5 2 28 116 52 97 10 2 07 311 64 5? 1952 9 4 11 125 57 08 10 2 07 3?1 66 60 1951 7 3 20 132 60 ?7 10 2 07 331 6B67 1950 6 2 74 138 63 01 .3 2 70 344 71 37 1949 8 3 65 146 66 67 11 2 28 355 73 65 1948 2 0 91 148 67 58 U 2 ?3 366 75 93 1947 6 2 74 154 70 32 9 1 B7 375 77 BO 1946 4 1 83 158 72 15 5 1 04 380 78 84 1945 1 - 46 159 72 60 2 0 41 38? 79 25 1944 1 0 46 160 73 06 3 0 6? 385 79 88 1943 2 0 91 16; 73 97 6 1 24 391 81 1? 1942 4 1 83 166 75 80 4 0 83 395 81 95 1941 4 1 83 170 77 63 8 I 66 403 83 61 1940 5 2 ?a 175 79 91 7 1 45 410 85 06 1939 2 0 91 177 80 82 10 2 07 420 B7 14 1938 1 0 46 178 81 28 1 0 21 4?1 87 34 1937 1 0 46 179 81 74 5 1 04 4?6 88 38 1936 1935 2 0 91 1B1 82 65 4 0 83 430 89 21 1934 1 0 46 IB? 83 U 2 0 41 432 89 63 1933 3 1 37 185 84 47 4 0 83 436 90 46 1932 1 0.46 186 84 93 2 0 41 438 90 87 1931 2 0 91 188 85 84 4 0 83 442 91 70 1930 1 0 46 1B9 86 30 1 0 21 443 91 91 1929 2 0 91 191 87 21 2 0 41 445 92 3? 1928 1 0 46 192 8767 1 0 21 446 92 53 1927 2 0 91 194 88 58 2 0 41 448 92 95 1926 4 1 63 198 90 41 5 1 04 453 93 98 1925 1924 1 0 46 199 90 87 3 0 6? •156 94 61 1923 1 0 46 200 91 32 1 0 21 457 94 81 1922 1 0 46 201 91 78 1 0 21 458 95 02 1921 1920 2 0 91 203 92 69 2 0 41 460 95 44 1919 1 0 46 ?04 93 15 1 0 21 461 95 64 1918 1917 1916 1915 1914 1 0 46 ?05 93 61 1 0 21 462 95 85 1913 1912 1911 1910 1909 1908 1907 1906 1 0 21 463 96 06 1905 1 0 46 206 94 06 1 0 ?1 464 96 27 1904 1 0 46 ?07 94 52 1 0 21 465 96 47 1903 1 0 46 208 94 98 1 0 21 466 96 66 1902 1901 I 0 21 467 96 89 1900 1 0 46 209 95 43 1 0 21 468 97 10 1899 1698 1 0 46 210 95 89 1 0 21 469 97 30 1697 1896 1895 1 0 46 211 96 35 1 0 21 470 97 51 1694 1893 1 0 46 212 96 60 1 0 21 471 97 72 1892 1 0 21 472 97 93 1891 I 0 46 213 97 26 1 0.21 473 98 13 1690 1669 1688 1687 1 0 21 474 98 34 1686 1 0 46 ?14 97 7? 1 0 21 475 9B 55 1665 1 0 46 ?15 98 17 2 0.41 477 98 96 1684 1683 1882 1681 ? 0 91 217 99 09 2 0.41 479 99 36 1680 1679 1678 1 0 46 218 99 54 1 0.21 480 98 59

1855 1 0 46 219 100 00 2 0.41 48? loo 00 178

Table 59

Number of Years Required to Give 50$, 75$, and 90$ of Cita­ tions by Section

Total T/I R/D Exp number of years from I966 required to give 50$ unique citations 5 4 5 12 75$ unique citations 12 9 10 24 90$ unique citations 26 16 19 40 50$ gross citations 4 3 3 9 75$ gross citations 10 7 9 18 $ gross citations 23 14 17 33

In each case, the addition of the repeat citations decreased the number of years required to achieve the given percentage, indicating that the repeat citations are primarily from the most recent years, and are not taken proportionately from all the years cited. It can also be seen that the number of years required to achieve the given percentages is much greater for the Exp than for either the T/I or the R/D. This would indicate that the citations from the Exp are spread over a larger number of years compared to those from the T/l and the R/D which are concentrated in the more recent years. One reason for this would be the fact that the experimental section 179 deals primarily with chemical compounds, their synthesis and

identification. These properties not only do not change

once (correctly) determined, but also the information is readily

indexed and retrieved in bibliographic tools through the

chemical compound name or formula. This is in comparison

with the interpretive material likely to be referenced in the T/l

and R/D which not only can change with time as new interpreta­

tions, new phenomena, etc. are discovered or described (there­

fore older material is no longer cited) but also is more

difficult to index, both for content and consistency over

the years, and therefore more difficult to retrieve.

The citations from the T/l tend to be very slightly more

concentrated in the recent years than citations from the R/D,

since the number of years required to achieve the given per­

centage is lower. Thus it would appear that the author is

citing primarily recent work as related background and intro­

ductory material. This is somewhat contradictory to the his­

torical presentation, reaching back in time, which might be

expected; the idea of a comprehensive retrospective search which is the basis for many information storage and retrieval

systems.

The results might be partially due to the influence of source articles published as part of a series, where it might 180 be expected that only recent related articles would be cited in the T/l rather than repeating citations to somewhat older articles which have already been cited in earlier numbered articles in the series. As mentioned before, when discussing the self citation, researchers tend to continue working and publishing in specific areas of interest. Therefore, even if the source article is not labeled as part of a series, it might be expected that each and every publication by an author will not give an extensive historical background but will, in effect, build on introductory material presented in his earlier articles.

Another factor causing citations from the R/D to be slightly less concentrated in the most recent years, is that the R/D is presenting, as well as interpreting, experimental data and procedures, and therefore would experience some spreading in cited years as a result of these experimental type citations.

Tables 55, 56, 57, and 58 also show that the citations from the Exp are distributed among a larger number of years than those from the R/D, which, in turn, is larger than that for the T/l (59:48:38 unique citations; 63:51:38 gross citations respectively). However, it is realized that this is directly related to the sample size, the total number of citations, 181 and if a large enough sample of citations were studied, the number of different years cited might eventually become the same for each section.

It would be more meaningful perhaps, to look at the number o£ citations and the corresponding percentage for each section over given periods of time. These data are presented in Table 60. The addition of the repeat citations increases' the percentage of citations for all three sections for the time period I960-I966. In most other cases, the addition of the repeat citations is not enough to compensate for the total increase in citations and therefore the percentage decreases.

Again, this would indicate that the repeat citations are primarily from the most recent years. This also supports the idea that the citations from the T/l are more concentrated in the most recent years, I96O-I966, than are those from the

R/D (67.34$ : 61.08$ unique citations; 71.91$ : 68.84$ gross citations). However, after i960 the R/D has the larger per­ centage until the very early years are reached when once again the T/I has the larger percentage. The wider time spread of the citations in the Exp is also illustrated by the data in

Table 60.

The data from Tables 55, 56, 57, and 58 were also plotted in log-log form in Figures 9, 10, 11, and 12 to show graphically Table 60 182

Time Distribution by Section by Decade

Total T/I R/D Exp unique citations 1960-1966 607 233 317 57 $ 56.00 67.34 61.08 26.03 1950-1959 290 79 130 81 $ 26.75 22.83 25.05 36.99 1940-1949 81 8 36 37 $ 7.47 2.31 6.94 16.Q9 1930-1939 45 10 21 14 $ 4.15 2.89 4.06 6.37 1920-1929 22 2 6 14 $ 2.03 O.58 1.17 6.39 1910-1919 8 3 3 2 $ 0.74 O.87 O.58 0.91 1900-1909 13 6 3 4 $ 1.20 1.73 O.58 1.83 before 1900 18 5 3 10 $ 1.66 1.45 O.58 4.57 gross citations I960-I966 1203 361 662 180 $ 61.47 71.01 69.04 37.34 1950-1959 473 98 211 164 $ 24.17 19.52 21.69 34.02 1940-1959 121 8 47 66 $ 6.18 1.59 4.83 13.69 1930-1939 70 11 26 33 $ 3.58 2.19 2.67 6.84 1920-1929 29 3 9 17 $ 1.48 0.60 O.92 3.53 1910-1919 21 9 10 2 $ I.07 1.79 1.03 0.42 1900-1909 15 6 3 6 $ 0.77 1.20 0.31 1.25 before 19 00 25 6 5 14 $ 1.28 1.20 0.5I 2.90 183 1000-

100-

io-

T 10 too Figure 9. Time Distribution (Log-Lof Form) of Total Unique (Lower Curve) and Total Gross (Upper Curve) Citations

too

io--

i T 10 too Figure 10. Time Distribution (Log-Log Form) of T/l Unique (Lower Curve) and T/l Gross (Upper Curve) Citations 184

100 ;

io ••

Figure 11. Time Distribution (Log-Log Form) of R/D Unique (Lower Curve) and R/D Gross (Upper Curve) Citations

too-* ^

io-

Figure 12. Time Distribution (Log-Log Form) of Exp Unique (Lower Curve) and Exp Gross (Upper Curve) Citations 185 the distribution of citations with respect to time. (The number of citations for I966 and I965 were combined to com­ pensate somewhat for the lag period.) Although more irregular than the similar journal distribution plots, the data still produce a similar series of"curves.

The same models and the same computer program were then used as for the journal distribution to perform a least squares calculation in order to obtain the best-fitting straight line for the equation log C = log a + b log T where C = the number of citations, T - (I966 - year), log a = constant (C intercept), and b = constant (slope).

Since the sample was not large enough to produce a value for each year for each section, and since the plot of log C vs log T becomes discontinuous when C = 0, some correction factor was necessary. Therefore, for the purposes of compari­ son, it was decided to approximate the curves and calculate the least squares by changing C = 0 to C = 1. (Another alternative would have been to ignore the points when C = 0; however, it was felt that this would lead to a larger dis­ tortion than the chosen method.) In Figures 9, 10, 11, and 12, the data have been plotted only until the curves become dis­ continuous; however, all years back to I878 (T = 88), the earliest year of citation (disregarding I855), were considered 186 by the computer. The results of the least squares calcula­ tion are given in Table 61 along with the calculated S .

Table 61

Least Squares Calculation for Time Distribution by Section

Total T/I R/D Exp unique citations b - 1.470 - 1.013 - 1.230 - 0.786 log a 2.773 1.783 2.206 1.453 a 592.9 60.67 160.7 28.38 error ' 0.04413 0.04322 0.03397 0.04021 S * 0.210 0.208 0.184 0.201 gross citations b - 1.629 - 1.094 - 1.416 - 1.168 log a 3.140 1.949 2.576 2.177 a I38O 88.92 376.7 150.3 error O.0687I O.O6072 0.04717 0.04770 S O.262 0.246 0.217 0.218

The resulting best-fitting straight lines, the correspond­ ing equation, and the standard error of estimate have been added to Figures 9, 10, 11, and 12. It is to be understood that these best-fitting straight lines do not exactly fit the data, but are close approximations and can be used for comparative purposes. 187 Foi^the total and for all three sections, the slope of

the calculated straight line increases (negatively) when the

repeats are added, once again indicating that the repeat

citations are being added primarily from the most recent years.

In all cases, the plotted values for T = 1 (I965) are below the calculated values, illustrating the effect of the lag period. This lag period is due to the length of time required to write up the results of the research and have the report published. Very few citations will, or can be, made to articles which appear during this time period. This lag period was partially taken into account by combining the number of citations from I966 and I965.

In comparing the slopes of the calculated straight lines for the three sections, it can be seen that, in each case

(unique citations and gross citations), the slope for the experimental is the smallest (negatively) or flattest, indicating that the number of citations falls off less rapidly with time than it does for the T/l or the R/D.

However, in comparing the slopes of the calculated straight lines for the T/l and the R/D, the T/l is the smaller (nega­ tively) . Although the T/l has a slightly higher concentration of citations in the most recent years, the wider spread of the citations through the years produces a smaller slope or 188 a less rapid decrease in the number of citations over the total time period.

The cumulations of the number of citations from Tables

55, 56, 57» and 58 was also treated in a similar manner as

for the journal distributions; the data were plotted in semi-log form in Figures 13, 14, 15, and 16. The same model and the same computer program were used to perform a least squares calculation in order to obtain the best-fitting straight line for the equation CC = a + b log T, where CC = cumulated number of citations, T = (I966 - year), a = constant (CC intercept), and b = constant (slope). Since CC always has a positive value, there is no need for a correction for the years with no citations. (The use of cumulated values also produces a smoother curve and is often used for this very purpose.) The results of the least squares calculation are presented in Table 62 along with the calculated S .

The corresponding equation for the best-fitting straight line and the standard error of estimate have been added to

Figures 13, 14, 15, and 16.

It is interesting to note that the plotted curves for the T/I and the R/D all show a definite change of direction around T = 12 (1954) and two calculated straight lines would have resulted in a much better fit. The effect of WW II can 189

cOOOr

IOC

Figure 13. Cumulated Time Distribution (Semi-Log Form) of Total Unique (Lower Curve) and Total Gross (Upper Curve) Citations

100 ' <• u

Figure 14. Cumulated Time Distribution (Semi-Log Form) of T/l Unique (Lower Curve) and T/l Gross (Upper Curve) Citations 1000-

400

500- • -

100

Figure 15. Cumulated Time Distribution (Semi-Log Form) of R/D Unique (Lower Curve) and R/D Gross (Upper Curve) Citations

100

Figure 16. Cumulated Time Distribution (Semi-Log Form) of Exp Unique (Lower Curve) and Exp Gross (Upper Curve) Citations 191

Table 62

Least Squares Calculation for Cumulated Time Distribution by Section

Total T/I R/D Exp unique citations b 425.541 109.255 190.608 125.677 a 3I8.O56 150.538 183.365 -15.848 error 2360.409 321.406 718.614 99.999 S 48.58 17.93 26.81 10.00 y gross citations b 690.973 144.536 302.937 243.508 a 722.289 243.979 441.418 36.879 error 8045.713 659.317 2371.568 536.504 S 89.70 25.68 48.69 23.16 y

also be seen in a flattening of the curves around T = 22

(1944) although it is less noticeable in the R/D; instead, another change in direction appears around T = 18 (1948).

In each case, the addition of the repeat citations increases the value of the slope of the calculated straight line. By inspection, it can be seen that the larger slope is due to a sharper rise in the most recent years (low values of T) after the addition of the repeats, again indicating that proportionately more of the repeat citations are to these recent years. 192

As can be seen by comparisons of the standard error of

estimate, these calculated best-fitting straight lines do not

fit the data as well as those produced for the journal distri­

butions. The journal distribution has the advantage that

journals do not have a built-in order as does the time distri­

bution with the years. Thus the journals can be put into any

order desired, with the usual choice being from the journal

with the largest number of citations to the journal with the

smallest number of citations, thereby producing a smooth curve.

If some set order for journals existed, jagged curves similar

to those for the time distribution would result; or if the

years could be ordered from most cited to least cited then

smoother curves would result and there would be no years with no citations. Journals which are not cited are not considered; however, some one year has to be chosen as the cut-off date

in order to compare the time distribution data for each

section. This chosen cut-off date will also affect the

slope of the curves obtained.

Summary. Five distributions of citations have been studied in this investigation with respect to the three sections (the title/introduction (T/l), the results/ discussion (R/D), and the experimental (Exp)) of source journal articles reporting original research in the field of 193

organic chemistry, (1) the frequency distribution of the

citations, (2) the citation level distribution of the

citations, (3) the self citation and same journal citation

distribution of the citations, (4) the journal distribution of the citations, and (5) the time distribution of the citations. In the first three studies, the data were separated for the two source journals, the Journal of the

American Chemical Society - organic section and the Journal of Organic Chemistry, in order to determine if the relation­ ship between the three sections and the citation property varied with the choice of source journal. In all five studies, the number of unique citations (the number of biblio­ graphic citations) and the number of gross citations (unique citations plus repeat citations) were considered separately to illustrate the effect of the repeat citation.

It has been shown by a Chi Square test that the frequency distribution of the citations changes significantly (at the 0.01 level) with the addition of the repeat citations, both for the combined sample and for each source journal. When the repeat citations are added, the percentage of citations in the T/l decreases while the percentage of citations in both the R/D and Exp increases. The length of material (number of columns)in each section of the source article was also 194

considered in order to determine the concentration of citations

in each section. The T/I was found to have the highest concen­

tration. It was also found that JACS had, on the average, more citations, longer sections, and a higher concentration

of citations than JOC. One major exception was that JOC had

the larger number of columns of material in the Exp.

It has been shown by a Chi Square test that a significant

relationship (at the 0.01 level of significance) does exist between the citation level, measured in this study as (1) the whole of the cited article, (2) a part of the cited article, and

(3) word/words from the cited article, and the section of

the source article in which the citation occurs. When parts

from the experimental section of the cited article (usually a paragraph of synthesis or identification of a specific chemical

compound) were added to the word/words to form a new citation level distribution, whole-part-word/words*, a larger Chi Square value was obtained. Inspection of the contingency table revealed that the citation level distribution was as expected with the

T/l-whole, R/D-part, and Exp-word/words* cells having many more citations than expected and the T/l-word/words* and the Exp- whole having fewer citations than expected. For both citation level distributions, the addition of the repeat citations produced a larger Chi Square value. While the Chi Square values 195

were smaller for each source journal than for the combined

sample, a similar relationship between level and section was

found. (The smaller values can be related to the sample size.)

When only the unique citations were considered, the self

citation distribution was shown by a Chi Square test to be

significantly related to the section (at the 0.01 level of

significance), with more citations than expected in the T/l

and fewer than expected in the Exp. However, with the addition of the repeat citations, this relationship disappears (and the

Chi Square value is not significant even at the 0.80 level of significance), thereby illustrating the bias which would be introduced if only the unique citations were considered. These relationships were shown to be true for each of the source journals as well. The same journal citation distribution was shown to be not related to the section.

The journal distribution of the citations was studied in several different ways. It has been shown that the number of cited journals needed to obtain a given percentage of citations varies with the section, that the slope of the calculated (least squares) best-fitting straight line using a simple model for the plot of the number of citations (and the cumulated number of citations) against the journal rank order varies with section, and that the ranking of the 4-12 196 most cited (combined total sample of unique citations) journals is significantly different in the sections (by a Kendall coeffi­ cient of concordance W test, at the 0.01 level of significance).

In addition, it has also been shown that the repeat citations are primarily to journals already the most cited in the section.

The time distribution was also shown to vary with the section, both the number of years required to produce a given percentage of citations and the slope of the calculated (least squares) best-fitting straight line using a model similar to that used for the journal distribution. Again, the repeat citations were shown to be primarily to the years already the most cited in the section. 197

SUMMARY AND CONCLUSIONS

Summary

Most uses made of existing citations, the citation-tool

(as subject indication) and the citation-count (as numerical measurement), are based on the assumption that authors tend to cite articles in some constant manner, and that the relationship between the source article and -the cited article does not vary with one author or among many authors. However, relatively little is actually known about citation habits. The purpose of this study was to investigate the proposal that the relation­ ship between the cited article and the source article (in the literature of a particular field of science) is not constant, that variations in the use of citations do exist.

One such relationship might be that between the level (or amount) of material actually being cited by the source article and the section of the source article in which the citation occurs. Three different citation levels were differentiated on a qualitative basis (as opposed to a quantitative basis), the whole of the cited article, a part of the cited article

(sentence-paragraphs), and word/words from the cited article. (aw-- lyo

In addition, three sections of the source article were identified,

the title/introduction (T/l), the results/discussion (R/D),

and the experimental (Exp).

The specific hypothesis to be tested was then that cita­

tions from the T/l will tend to be to the whole of the cited

article, citations from the R/D will tend to be to a part of

the cited article, and citations from the Exp will tend to be

to word/words from the cited article. This relationship was

hypothesized because it was thought that properties of citations

in general would tend to vary with the section of the source

article in which the citation occurs, and citation level is a measurable property of citations.

The field chosen for study was organic chemistry since

some subject knowledge was necessary to make the judgments as

to what exactly was being cited. Two source journals were

chosen, the Journal of Organic Chemistry (JOC) and the Journal of the American Chemical Society - organic section (JACS).

Both contain reports of original research, including experi­ mental details, and have the required format with respect to the three sections chosen for study. The citations from twenty- five randomly chosen articles from each source journal from the year I966 were then studied. The study was confined to cita­ tions to other journal articles so that the amount of material 199 within each category of amount cited would be relatively

constant.

The level of material actually cited by the source article

was determined as outlined in the procedure. Statistical

analyses were then performed to see if a significant relation­

ship did exist between the citation level and the section of

the source article in which the citation occurs.

In addition, four other distributions of the citations were

also investigated, (1) frequency distribution, (2) self citation

and same journal citation distribution, (3) journal distribution,

and (4) time distribution. The journal and time distributions were studied in order to indicate that variations in citation properties do exist with respect to the section of the source article in which the citation occurs. For the frequency dis­ tribution, citation level distribution, and the self citation and same journal citation distribution, the data were separated for the two source journals to give some indication of the variations in citation properties which might be found even within source journals by the same publisher and therefore with the same editorial policies (in this case the American

Chemical Society).

Another important factor, which was recognized only after the collection of the data began, was that one bibliographic citation could be used several times, for different purposes, 200

either in the same section or another section. In addition to

the assumption of a constant relationship between the source

article and the cited article, citation uses, both the citation-

tool and the citation-count, usually recognize only one use of

each citation. However, since this study was concerned with

the identification of the amount of cited material and the

section of the source article, it was quickly realized that each and every use of a bibliographic citation had to be

considered. Therefore, throughout the study, two samples have,

in effect, been considered, the unique citations (each biblio­ graphic citation counted only once as in a traditional citation- count, the first time it was used by the source article) and the gross citations (all uses counted, unique citations plus repeat citations).

*i Frequency Distribution. The first aspect investigated was the frequency distribution of the citations among the three sections, and how this distribution compared with the amount of material in each section (as measured by the number of columns of material).

It was shown that the frequency distribution by section changed significantly between unique citations and gross cita­ tions, with the percentage of citations in the T/l decreasing and the percentage of citations in the R/D increasing with 201

the addition of the repeat citations. This is to be expected

from the format of the source article since the further on in

the article, the more citations there are available to be

repeated. From the gross citations, the percentage of citations

is about the same for the T/l and the Exp, each about one-half

the percentage of citations for the R/D. However, the concen­

tration of citations per amount of material in each section

was the greatest for the T/l, due primarily to the smaller

amount of material. This greater concentration is somewhat

to be expected since the purpose of the T/l is to present

background material and related research, all of which should

be cited.

The frequency distribution of the citations in each of

the two source journals was also studied. While the twenty-five

source articles from JACS tended to be longer than the twenty-

five source articles from JOC, this difference in length was not enough to account for the fact that JACS produced twice as many citations as JOC, as shown by the larger concentration of

citations in JACS. The frequency distribution of the citations with section also differed in the two source journals, with JOC having both a higher percentage of citations and, in fact, more source material in the Exp than did JACS, while JACS had the higher percentage of citations in the R/D. It was suggested 202

that this might indicate a slight difference in emphasis in the

two journals; articles submitted to JACS might tend to emphasize

the presentation and discussion of results, while those sub­

mitted to JOC might tend to emphasize synthetic procedures.

Citation Level Distribution. The basic purpose of the

study was accomplished, to indicate that a significant relation­

ship does exist between the level of material from the cited

article and the section of the source article in which the

citation occurs. When the cited parts from the experimental

section of the cited article (primarily synthesis-identification

of specific chemical compounds) were added to the word/words

category to form a new citation level, word/words*, an even more

significant relationship was shown to exist. Inspection of the

data revealed that the relationship did lie in the expected

direction with the number of T/l-whole, R/D-part, and Exp-

word/words* citations being much larger than expected and the number of T/l-word/words* and Exp-whole citations being much

smaller than expected if no relationship existed between

citation level and the section.

The addition of the repeat citations also produced a more significant value for each of the whole-part-word/words

and whole-part-word/words* distributions indicating that the

distribution of the repeat citations was not the same as 203

that for the unique citations, with most of the repeat citations being to parts and word/words, a very few to the whole of the cited article.

Similar significant relationships were also found in each case for both of the source journals.

It was suggested that this relationship exists since citations in the T/l are most likely to be from general state­ ments about the research topic of the source article to articles which are quite broadly related to this topic, citations in the R/D are most likely to be from specific stages within the research to similar specific stages within other articles, while citations in the Exp are most likely to be from synthesis/identification or specific chemical compounds used in the research to synthesis/identification of the same chemical compounds.

Self Citation, Same Journal Citation Distribution. The phenomena of the self citation and the same journal citation were also investigated with respect to the section of the source article, both before and after the addition of the repeat citations, and for each source journal. It was found that the self citation was significantly related to the section for the unique citations, due to a higher than expected concen­ tration in the T/l. However, when the repeat citations were 204

added, there was no significant relationship. Thus, the

author does tend to cite himself proportionately in all three

sections, and if the repeat citations are not considered, a

false distribution pattern is obtained.

Over one-third of the repeat citations were self citations.

It was suggested that this high percentage might be expected

since researchers tend to continue working in specific areas

of interest and therefore the former research, most related to

the general topic of the source article, the individual steps

within the research, and the specific chemical compounds dis­

cussed, which the researcher is sure to know about, are quite

often his own.

No significant relationship was evident between same

journal citations and the section of the source article, either

before or after the addition of the repeat citations. There

was some indication that the two source journals did have a

tendency towards same journal citation, since JACS had a larger

percentage of citations to JACS (same journal) than did JOC,

and similarly JOC had a larger percentage of citations to JOC

(same journal) than did JACS.

Journal Distribution. This distribution of the journals

cited, the number of citations per journal arranged in rank order from the most cited journal to the least cited journal 205 was also investigated to see if any differences existed between the citations in the three sections. The differences were not as great as appeared with the citation level distribution, but some differences were evident.

The first aspect noted was that a different number of journals were cited in each section. For the gross citations, the R/D had the largest number of journals cited as well as the largest concentration of citations per journal. While the

T/l and Exp had about the same number of gross citations, the

Exp had fewer cited journals and therefore had the higher con­ centration of citations per journal.

From the journal distribution, it was evident that the repeat citations were primarily to journals already heavily cited; for each section, when the repeat citations were added, a smaller percentage of journals was required to reach a given percentage of citations.

The same exponential journal distribution pattern was obtained for each section, with a large number of citations to a few journals, and a small number of citations to many journals.

Two simple models for the journal distribution were used to calculate best-fitting straight lines to fit the data so that some comparisons could be made. More sophisticated models have been proposed; however, the emphasis in this study was to show 206

some evidence of differences and it was felt that the simple

models would be appropriate. The first model was that for the

number of citations plotted against the journal rank order, the

second the cumulated number of citations against the journal

rank order. Differences in the slope of the calculated

best-fitting straight line were obtained for the three sections

using both models, with the slope being the largest for the

R/D in each case. The models used fit the T/l data the best.

Many of the differences in the journal distribution with

section however, can be attributed to the sample size since the number of journals cited in each section does not increase pro­ portionately with the number of citations. This may account for

the differences between the R/D and the other two sections. How­

ever, the T/l and the Exp sample sizes were comparable and dif­

ferences between these two sections can be identified. One primary difference is the smaller number of journals cited in the Exp. It was suggested that this might be due to the fact that not all journals contain experimental details (e.g., review articles, communications) and therefore the Exp data are obtained from a smaller population of journals.

Another aspect of interest in the ieurnal distribution was the order in which the journals were ranked in each section.

In all cases, JACS, JOC, and the Journal of the Chemical Society 207

were ranked 1, 2, 3 respectively. Comparison of the ranks of

the nine next most cited journals (by total unique citations)

indicated that they were ranked in random order by the three

sections, both for unique citations and gross citations. Com­

parison of the ranks of these twelve most cited journals within

each section indicated that the rank order does not change

significantly with the addition of the repeav citations, again

supporting the idea that the repeat citations are primarily to

journals already heavily cited in the section.

Time Distribution. Finally, the time distribution of the

citations with respect to the three sections was investigated

and again some differences were observed.

Similar exponential time distributions were obtained for

each section as with the journal distribution^- -However,

the distribution was not as smooth, since the years have a built-in order which is not present with the journals. As with

the journal distribution, the repeat citations were taken from

the already most cited years (the most recent years). The

same simple distribution models from the journal distribution were also used for the time distribution by replacing the journal rank with the number of years lapsed from the publication date of the cited article (1966-year). 208

The greatest difference could be seen in the citations from the Exp which were spread over the largest number of years; more years were needed to obtain the given percentage of cita­ tions than for the T/l or the R/D. Similarly the slope of the calculated best-fitting straight line was the smallest (nega­ tively) for the Exp (number of citations against years lapsed), again indicating that the citations from the Exp do not tend to be as concentrated in the most recent years as those from the

T/I and the R/D.

More surprising, however, was the fact that the citations from the T/l were more concentrated in recent years than those • from the R/D, contrary to the idea of the historical retrospec­ tive search of the literature conducted before the beginning of a research problem (and the basis of many information storage and retrieval systems). Citations from the T/l however, did tend to be spread over a wider range of years than the R/D.

It was suggested that, in the pure research field of organic chemistry, researchers tend to continue to work in specific areas of interests (some indication of this is the number of source articles that are labeled as part of a series); therefore each publication might not give a complete historical account, but will build on previous papers by the same author. It is also suggested that both the author and his intended audience 209 already have a great deal of knowledge in common so that an article need not start from the historical beginning every time.

Therefore, only recent articles of interest will be discussed in the T/l. There will, however, be some older articles related to the specific topic covered in the source article which may need to be cited, accounting for the wider range of years.

In all three sections, there was some indication in the curves for the cumulated time distribution of the existence of two curves (often considered to be *n indication of two different types of literature). Again, these were much less distinguishable in the Exp. The effects of World War II on publication could also be identified.

Conclusions

It is concluded that the citation level does vary signifi­ cantly with the section of the source article in which the citation occurs, since citation properties in general tend to vary with the section. It was also demonstrated that citation level was a measurable property of citations. Differences in citation property distributions between two closely related source journals have been shown to exist. Finally, the importance of the repeat citation has been emphasized and some 210

indication of the bias which can be introduced if it is

ignored has been illustrated.

Research Stage. Some tentative suggestions have been made

as to why the specific relationship between the section and the

citation property exists. A more general reason proposed for

the differences in properties among the citations from the

three different sections of the source article is that the

three sections represent three different needs of the researcher.

Most use studies now recognize that the «.:eed of the

researcher for the information is an important factor and can

not be overlooked, that different needs will be satisfied in

different ways. Straight citation-counts as some measure of

literature use do not take into account any such factor. It

is suggested in this study that the section of the journal

article presenting the results of original research can be used

to approximate one indication of different user needs.

Voigt identified three major needs of the scientist, the

current (keeping up with the literature), the everyday (specific

information on specific occasion), and the exhaustive (retro- 1 spective search). Hanson also discussed user studies and 2 concluded that these same three major needs had been identified.

Menzel added what he considered to be a fourth major need, that 211 of brushing-up in related areas, in addition to several other minor needs.

It might be suggested that the three major needs might be related to the sections of the journal article, where the T/l might ideally indicate the exhaustive approach, the Exp might indicate the everyday approach, and the R/D might indicate the current approach. However, it is felt that the three sections more closely approximate the three research stages of formula­ tion of the problem (T/l), conducting the research (Exp), and analyzing the results (R/D).

Recently, librarians and information scientists have been questioning several previously held assumptions concerning the relationship of a document to some problem and whether or not this relationship is a constant and whether it can be deter­ mined or measured. This assumption is directly related to the problems of relevance, indexer inconsistency, and citations, since all are involved with the relationship between a document and some problem, viz., a given stated problem, some future expected problem, and a problem as indicated by interest in some other document respectively.

The work done by Rees on the problem of relevance is particularly pertinent since he considered the research stage to be an information need. He identified three stages of 212

research, RS-. Formulation of the research problem; RS,:

Implementation of experimental approach and design; RSp:

Evaluation of data; and found that the determination of the

relevance of a document to a stated problem is significantly k related to the stage of research described to the judges.

If the sections of the source article can be accepted as

being an indication of the research stage, then the differences

in the citation properties with respect to the section can be

attributed to the different needs which the researcher has at

different stages in his research.

However, reasons for the particular differences in cita­

tion properties with respect to section which can be identified

can only be suggested (with occasional support from other authors who have also made suggestions) since the whole process of the citation is so complex and so little is known. The problem includes such steps as: what information the researcher needed; what information he wanted (thought he needed); what information was available in the primary literature; what secondary literature was available leading to the primary literature; what information was available directly in the secondary literature; how the researcher looked for the information; how he found the informa­ tion; whether it was the best information available; whether 213

he actually used the document cited; whether he cited all

documents he used; and finally why he cited this particular

information.

Level of Indexing. The idea of the variation in the level

of material actually being cited has some implications for

indexing policy. Perhaps, instead of talking in the more

general terms of "depth" of indexing, where depth indicates

only the number of terms assigned to the document, more atten­

tion might well be paid to the level or amount of material within

the document which should be indexed. In his study of indexing

consistency at various indexing depths, Harris suggested that

the problem of the size of the unit to be indexed (sentence,

paragraph, whole article) was one source of indexer inconsistency.

Most subject indexing is done on the basis of the intrinsic

content of the source document. However, as Lancaster states,

"efficient subject indexing is not necessarily achieved by

labeling a document on the basis of its intrinsic subject matter.

Rather it is achieved by labeling a document according to the

types of users who may be expected to derive most benefit from

it." It was hoped, in the original proposal for this study,

that the indexing terms used by Chemical Abstracts for the cited

articles could be obtained so that some comparison could be made between the actual use of the cited article (as repre- 214

sented by citation in the source articles) and the available indexing (as represented by Chemical Abstracts). It was sug­ gested that indexing done on the basis of the intrinsic content of the source article would be more suitable for uses made of

citations in the T/l (historical background - article) and Exp

(specific chemical compounds - word/words, parts from the experimental section of the cited article) than in the R/D

(analysis and interpretation of results - parts from the cited article). However, this information was not readily available.

While it was not possible to gather data for a statistical analysis, inspection of the large number of citations examined does lead to some suggestions concerning indexing problems.

The inference of the distribution patterns described might indicate not only that an optimum level of indexing is to be defined for each of the various expected uses for the article, but that this optimum level may well vary through time since the use (as defined in this study by the section of the article) made of information appears to vary with time.

Thus a document might be indexed at several different levels with several different user needs considered, and this indexing might be made available in different forms at dif­ ferent times. 215

This is somewhat related to the modular idea of providing access to a document through various channels. One example of this is the work being done by Chemical Abstracts Service with the aid of computers in the area of multiple access. At present, several different services are being provided from the same information input: Chemical Titles (a KWIC index),

Chemical Abstracts, and Chemical and Biological Activities

(a specialized journal which uses some of the information from Chemical Abstracts). Other access points to chemical articles are provided by Current Contents and Science Citation

Index.

Suggestions for Further Research

If the citation level were not investigated, a much larger sample of citations could have been chosen and, perhaps, greater differences in the time distribution and the journals distribu­ tion with respect to the sections of the source article might have emerged. The language distribution and the country dis­ tribution of the citations, often studied in citation-counts, might also be investigated as citation properties in a larger study.

A different sampling method might be used, one based on the number of citations, rather than the number of articles 216

(emphasis on different uses by different authors, rather than

different uses by the same authors), i.e., a few citations from

each of many articles.

With the availability of Science Citation Index, a cited

article could be traced to see whether it is being used (as

measured by section of the source article, level of material

cited, etc.) in a similar manner by different source articles.

Another aspect to investigate might be differences in the uses

of citation that could be attributed to the different source journals. Citations could also be studied through time to see

if their use by source articles changes with time.

Another area of research would b« to look at the indexing available for the cited article in conventional subject heading indexes, KWIC indexes, etc. to see if some comparison could be made between this available indexing and the actual use made of the cited article by the source article.

The use of the available indexing for the cited article, without referring to what was actually cited, would be some­ what similar to a study of subject dispersion. From several suggestions made in this study, it might be expected that citations in the T/l would more likely be to articles closely related to the source articles, and to each other, than citations in the Exp. 217 It might also be suggested that an indication of the section

of the source article in which the citation occurs could there­

fore be used in some way to improve citation indexes. For

example, it might be found that, in the field of chemistry,

citations in the Exp could be omitted from citation indexes

since the cited article is likely to be unrelated to the topic of the source article, and the indexing of experimental data concerning chemical compounds is readily available from other sources.

In the discussion of the format of the source articles and

their citations, the phenomena of the extra citation (more than one bibliographic citation listed in one footnote) and the multiple citation (more than one superscript given together) were mentioned. The use of two or more citations at a given point in the document might be investigated to see if these articles might not be more closely related to each other than to the other cited articles, a form of bibliographic coupling

(e.g., in several cases, the extra citations in a footnote were all by the same author).

It might also be useful to look more closely at the repeat citations (this study has indicated that they are primarily to parts and word/words from the cited articles, from the most cited journals, the most cited years, and are self 218 citations in over one-third of the time) and the bias which might be introduced if they are ignored. Again, a closer relationship might exist between citations which are re­ peated several times and the source article than between citations which are used only once and the source article.

The whole problem of why the scientist makes citations, why he makes his particular citations, and how they reflect or do not reflect his actual research and use of the literature needs much more study.

And finally, one very interesting area for future research would be a historical study of the origins and development of the citation in the scientific journal literature. 219 Footnotes

' Melvin J. Voigt. Scientists' Approach to Information. Chicago, ACRL, I96I. (ACRL Monograph No. 24). o C. W. Hanson. Research on Users' Needs: Where is it Getting Us? Aslib Proceedings, 16, 64-78 (1964). 3 Herbert Menzel. Review of Studies in the Flow of Information Among Scientists. New York, Columbia University, Bureau of Applied Social Research, i960. 2v. h Alan M. Rees and Douglas G. Schulz. A Field Experiment Approach to the Study of Relevance Assessments in Relation to Document Searching. Cleveland, Ohio, Case Western Reserve University, Center for Documentation and Communication Research, I967.

^Dean Harris, W. Boyd Rayward, and Elain Svenonius. The Testing of Inter-Indexing Consistency at Various Indexing _. Depths. Chicago, University of Chicago, Graduate Library School, I966. (Studies of Indexing Depth and Retrieval Effectiveness. Working Paper No. 38O-2).

F. Wilfrid Lancaster. Information Retrieval Systems; Characteristics, Testing, and Evaluation. New York, Wiley 1968. p. 3. 220

BIBLIOGRAPHY

Adams, W. Mansfield. Relationship of Keywords in Titles to References Cited. American Documentation, 18, 26-32 (I967).

Barber, A. Stephanie. A Critical Review of the Surveys of Scientists' Use of Libraries. In The Provision and Use of Library and Documentation Services, ed. by W. L. Saunders. Oxford, Pergamon Press, I966. pp. 145-179.

Barker, Dale Lockard. Characteristics of the Scientific Literature Cited by Chemists of the Soviet Union. Unpublished Ph. D. dissertation, Graduate School of Library Science, University of Illinois, I966.

Barnard, Cyril C. Bibliographical Citations. Librarian and Book World, 39., 105-110, I7I-I75, 125-129 (1950).

Barnard, Cyril C. The Selection of Periodicals for Medical and Scientific Librarian!!. Library Association Record, 40, 549-557 (1938). Barr, Larry J. Origin of Footnotes. RQ, £, I6-I7 (I966).

Bayer, Alan E. and Folger, John. Some-Correlates of a Citation Measure of Productivity in Science,, Sociology of Education, 3£, 381-390 (1966).

Boig, Fletcher S. and Howerton, Paul W. History and Develop­ ment of Chemical Periodicals in the Field of Organic Chemistry, 1877-1949. Science, 115, 25-31 (1952).

Bourne, Charles P. Some User Requirements Stated Quanti­ tatively in Terms of the 90 Percent Library. In Electronic Information Handling, ed. by Allen Kent and Orrin E. Taulbee. Washington, D. C, Spartan, I965. pp. 93-110.

Bradford, S. C. Sources of Information on Specific Subjects. Engineering, 132, 85-86 (1934). 221

Bradford, S. C. Documentation. London, Crosby, Lockwood, 1948. pp. 110-120.

Broadus, Robert N. A Citation Study for Sociology. American Sociologist, 2, 19-20 (1967).

Brodman, Estelle. Choosing Physiology Journals. Medical Library Association, Bulletin, 3£, 479-483 (1944).

Brookes, B. C. The Derivation and Application of the Bradford-Zipf Distribution. Journal of Documentation, 24, 247-265 (1968).

Brown, Charles Harvey. Scientific Serials. Chicago, ACRL, 1956. (ACRL Monograph No. 16).

Brown, Sanborn C. A Bibliographic Search by Computer. Physics Today, l£, 59-64 (I966).

Buckland, M. K. and Hindle, A. Library Zipf. Journal of Documentation, 2£, 52-57 (1969).

Buckland, M. K. and Woodburn, I. Some Implications for Library Management >' . Scattering and Obsolescence. Lancaster, Eng., Library, University of Lancaster, I968. (University of Lancaster Library, Occasional Papers No. 1).

Burton, R. E. and Kebler, R. W. The "Half-Life" of Some Scientific and Technical Literatures. American Docu­ mentation, 11, 18-22 (i960).

Cole, P. F, Journal Usage Versus Age of Journal. Journal of Documentation, If}, 1-11 (I963).

Cole, P. F. A New Look at Reference Scattering. Journal of Documentation, 18, 58-64 (I962).

Cole, Stephen and Cole Jonathan R. Scientific Output and Recognition: A Study in the Operation of the Reward System in Science. American Sociological Review, 32_, 337-390 (1967). Cuadra, Carlos A. Identifying Key Contributions to Informa­ tion Science. American Documentation, l£, 289-295 (1964). 222

Dahling, R. L. Shannon's Information Theory: The Spread of an Idea. In Studies of Innovation and of Communication to the Public. Palo Alto, Stanford University, Institute for Communication Research, I962.

Dalziel, Charles F. Evaluation of Periodicals for Electrical Engineers. Library Quarterly, £, 354-372 (1937).

Dennis, W. The Age Decrement in Outstanding Scientific Contributions: Fact or Artifact? American Psychologist, 13_, 457-460 (1958).

Deweze, Andre. Automated Establishment and Utilization of Bibliographical Citation Indexes. UNESCO Bulletin for Libraries, 18, 211-223 (1964).

Dutta, S. Literature Citations in Scientific and Technical Periodicals--A Survey. Journal of Scientific and Industrial Research (India), I7A, 259-261 (1958).

Egan, Margaret and Henkle, Herman H. Ways and Means in which Research Workers, Executives, and Others Use Information. In Documentation in Action, ed. by Jesse H. Shera and others. New York, Reinhold, 1956. pp. 137-159.

Fairthome, R. Library Zipf. Journal of Documentation, 2£, 152-154 (1969). Friis, Th. The Use of Citation Analysis as a Research Tech­ nique and Its Implications for Libraries. South African Libraries, 23_, 12-15 (1955).

Fussier, Herman H. Characteristics of the Research Literature Used by Chemists and Physicists in the United States. Library Quarterly, 1£, 19-35, 119-143 (1949). Garfield, Eugene. Citation Indexes for Science. Science, 122, 108-111 (1955).

Garfield, Eugene. Citation Indexes for Science, Science, 123, 62 (1956).

Garfield, Eugene. Citation Indexes in Sociological and Historical Research. American Documentation, 14, 289-291 (1963). 223 Garfield, Eugene. Citation Indexes--New Paths to Scientific Knowledge. Chemical Bulletin, 43_, 11-12 (1956).

Garfield, Eugene. Citation Indexing: A Natural Science Literature Retrieval System for the Social Sciences. American Behavioral Scientist, £, 58-6I (June, 1964).

Garfield, Eugene. Patent Citation Indexing and the Notions of Novelty, Similarity, and Relevance. Journal of Chemical Documentation, 6, 63-65 (I966).

Garfield, Eugene. "Science Citation Index"—A New Dimension in Indexing. Science, 144, 649-654 (1964).

Garfield, Eugene. Primordial Concepts, Citation Indexing and Historio-bibliography. Journal of Library History, 2, 235-249 (1967). Garfield, Eugene and Sher, I. H. ISI's Experiences with ACSA—A Selective Dissemination System. Journal of Chemical Documentation, £, 147-153 (I967).

Garfield, Eugene and Sher, I. H. New Factors in the Evalua­ tion of Scientific Literature Through Citation Indexing. American Documentation, 14, 195-201 (I963).

Garfield, Eugene, Sher, I. H., and Torpie, R. J. The Use of Citation Data in Writing the History of Science. Philadelphia, Institute for Scientific Information, 1964.

Garner, Ralph. A Computer-orientated Graph Theoretic Analysis of Citation Index Structures. In Three Drexel Informa­ tion Science Research Studies. Philadelphia, Drexel Press, I967.

Ghosh, J. S. Literature Searching with the Science Citation Index: An Experimental Study. Annals of Library Science and Documentation, 14, 133-142 (I967).

Gosnell, Charles F. Obsolescence of Books in College Libraries. College and Research Libraries, 5, 115-125 (1944). Great Britain Advisory Council on Scientific Policy. Survey of Information Needs of Physicists and Chemists. Journal of Documentation, 21, 83-112 (I965). 224

Groos, 0. V. Bradford's Law and the Keenan-Atherton Data. American Documentation, If), 46 (I967).

Gross, P. K. L. and Gross, E. M. College Libraries and Chemical Education. Science, 66, 385-389 (1927).

Gushee, David E. Reading Behavior of Chemists. Journal of Chemical Documentation, 8, 191-194 (1968).

Hanson, C. W. Research on Users' Needs: Where is it Getting Us? Aslib Proceedings, 16, 64-78 (1964).

Hodgson, James GL The Preparation of Bibliographies and Bibliographical Footnotes and Citations. Fort Collins, Colorado, Library, Colorado A & M College, 1956.

Huang, Theordore S. Efficacy of Citation Indexing in Reference Retrieval. Library Resources and Technical Services, 12, 415-434 (1968).

Janke, N. C. Abuses of Citation Indexing. Science, I56, 892 (1967). Jahoda, G. Information-gathering and Use Habits of Chemists. Journal of Chemical Documentation, 4, I53-I56 (1964).

Kaplan, Norman. Norms of Citation Behavior: Prolegomena to the Footnote. American Documentation, 16, 179-184 (I965). Keen, E. M. Citation Indexes. Aslib Proceedings, 16, 246-251 (1964).

Kendall, M. G. The Bibliography of Operational Research. Operational Research Quarterly, 11, 3I-36 (i960).

Kessler, M. M. Bibliographic Coupling Between Scientific Papers. American Documentation, 14, 10-25 (1963).

Kessler, M. M. Bibliographic Coupling Extended in Time: Ten Case Histories. Information Storage and Retrieval, 1, I69-I87 (1963). Kessler, M. M. Comparison of the Results of Bibliographic Coupling and Analytic Subject Indexing. American Documentation, 16, 223-233 (1965). 225 Kessler, M. M. Technical Information Flow Patterns. Western Joint Computer Conference, Proceedings, 247-257 (I96I).

Kilgour, Frederick (G. Publication of Scientific Discovery: A Paradox. American Documentation Institute, Proceedings of the Annual Meeting, 427-430 (19*66).

Kuppers, J. R. Literature Citation Counting. Science, 133. 1138 (1961).

Lebo, Dell. Publication Versus Communication. Science, 128, 424-425 (1958).

Leimkuhler, F. F. The Bradford Distribution. Journal of Documentation, 23_, 197-207 (I967).

Lipetz, Ben-Ami. Compilation of an Experimental Citation Index from Scientific Literature. American Documentation, 1£, 251^266 (1962).

Lipetz, Ben-Ami. Improvement of the Selectivity of Citation Indexes to Science Literature Through the Inclusion of Citation Relationship Indicators. American Documenta­ tion, 16, 81-90 (I965).

Lipetz, Ben-Ami. Evaluation of the Impact of a Citation Index in Physics. In International Federation for Documentation. Conference, 1965. Proceedings. Washington, D. C, Spartan, I966. pp. I07-II5.

Lubetzky, Seymour and Hayes, R. M. Bibliographic Dimensions in Information.Control. American Documentation, 20, 247-252 (1969). MacRae, Duncan. Growth and Decay Curves in Scientific Citations. American Sociological Review, 3_4, 63I-635 (1969). McAnally, A. M. Characteristics of Materials Used in Research in United States History. Unpublished Ph. D. disserta­ tion, Graduate Library School, University of Chicago, 1951. Magee, Marguerite. How Research Biochemists Use Information: An Analysis of Use of Information from Cited References. Unpublished M. A. dissertation, Graduate Library School, University of Chicago, I966.

Malin, Morton V. The Science Citation Index: A New Concept in Indexing. Library Trends, 16, 374-387 (1968).

Margolis, J. Citation Indexing and Evaluation of Scientific Papers. Science, 155, 121(3-1219 (1967).

Martyn, John. Examination of Citation Indexes. Aslib Proceedings, l£, 184-196 (I965).

Martyn, John. Bibliographic Coupling. Journal of Docu­ mentation, 20, 236 (1964).

Martyn, John and Glichrist, Alan. An Evaluation of British Scientific Journals. London, Aslib, I968. (Aslib Occasional Publication No. 1). » Matthews, G. M. and Van Luik, J. Citation and Subject Index­ ing in Science. Library Resources and Technical Services, £, 478-482 (I965).

May, Kenneth D. Abuses of Citation Indexing. Science, 156, 890, 892 (1967). Meadows, A. J. Citation Characteristics of Astronomical Research Literature. Journal of Documentation, 23, 28-33 (1967).

Menzel, Herbert. Review of Studies in the Flow of Informa­ tion Among Scientists. New York, Columbia University, Bureau of Applied Social Research, i960. 2 v.

Medawar, P. B. Is the Scientific Paper a Fraud? Tappi, 4£, 30A, 34A, 40A (1964).

Merton, Robert K. The Matthew Effect in Science. Science, 159, 56-63 (1968).

Merton, Robert K. Priorities in Scientific Discovery: A Chapter in the Sociology of Science. In The Sociology of Science, ed. by Bernard Barber and Walter Hirsch. New York, Free Press of Glencoe, I962. pp. 447-485. 227 Paisley, William J. The Flow of (Behavioral) Science Informa­ tion: A Review of the Research Literature. Palo Alto, Stanford University, Institute for Communication Research, I965.

Parker, Edwin B., Paisley, William J., and Garrett, Roger. Bibliographic Citations as Unobtrusive Measures of Scientific Communication. Palo Alto, Stanford Univer­ sity, Institute for Communication Research, I967.

Postell, William D. Further Comments on the Mathematical Analysis of Evaluation of Scientific Journals. Medical Library Association, Bulletin, 34,, 107-109 (1946). Price, Derek J. de Solla. Ethics of Scientific Publication. Science, 144, 655 (1964).

Price, Derek J. de Solla. Networks of Scientific Papers. Science, 14<9, 510-515 (1965).

Raisig, L. Miles. Mathematical Evaluation of the Scientific Serial. Science, 13JL, 1417-1419 (i960).

Rath, G. J. Research on the Management of Research and Development. In Second Congress on the Information System Sciences, 1964, ed. by J. Spiegel and D. E. Walker. Washington, D. C, Spartan, I965. pp. 435-444.

Rees, Alan M. and Schulz, Douglas G. A Field Experiment Approach to the Study of Relevance Assessments in Relation to Document Searching. Cleveland, Ohio, Case Western Reserve University, Center for Documentation and Communication Research, I967. 2 v.

Rees, Alan M. Relevancy and Pertinency in Indexing. American Documentation, 13_, 94-95 (1962).

Reif, F. The Competitive World of the Pure Scientist. Science, 134, 1957-1962 (1961).

Reisner, Phyllis. Pre-test and Potential of a Machine-stored Patent Index. In Kochen, Manfred. Some Problems in Information Science. New York, Scarecrow, I965. pp. 161-173. 228 Rinelander, Philip H. Notes on Footnotes. College and Research Libraries, 2£, 184 (I966).

Rutgers University. Graduate School of Library Science. Bureau of Information Sciences Research. Bibliography of Research Relating to the Communication of Scientific and Technical Information. New Brunswick, N. J., Rutgers-,* University Press, I967.

Salton, Gerald. Associative Document Retrieval Techniques Using Bibliggraphic Information. Association for Computing Machines, Journal, 10, 440-457 (I963).

Schoenbach, U. H. Citation Indexes for Science. Science, 123, 61-62 (1956).

Sheppard, Oden E. The Chemistry Student Still Needs a Reading Knowledge of German. Journal of Chemical Education, 12, 472-473 (1935).

Sher, I. H. and Garfield, Eugene. New Tools for Improving and Evaluating the Effectiveness of Research. Con­ ference on Research Program Effectiveness, I965. Proceedings. Washington, D. C, I965.

Spencer, Carol C. Subject Searching with Science Citation Index. American Documentation, 18, 87-96 (I967).

Stevens, Rolland E. Characteristics of Subject Literatures. Chicago, ACRL, 1953. (ACRL Monograph No. 6).

Stevens, Rolland E. The Use of Library Materials in Library Research: A Study of the Effect of Differences in Research Method. Unpublished Ph. D. dissertation, Library School, University of Illinois, 1951.

Steinbach, H. Burr. The Quest for Certainty: Science Citation Index. Science, 145, 142-143 (1964).

Sullivan, Frank S. A Garland of Ibids for Van Wyck Brooks. New Yorker, l£, 15 (April 19, 1941).

Symposium on Communication in Science: Documentation and Automation, London, I966. Communication in Science, ed. by A. de Reuck and J. Knight. London, Churchill, 1967. 229 Tagliacozzo, R. Citations and Citation Indexes: A Review. Methods of Information in Medicine, 6, 136-142 (I967).

Titley, Joan. Citation Upmanship. Rub-off, 16, 4-6 (I965).

Tukey, John W. A Citation Index for Statistics and Probability. Institut International de Statistique, Bulletin, 40, 747-756 (1964).

Tukey, John W. Keeping Research in Contact with the Literature: Citation indexes and Beyond. Journal of Chemical Documentation, 2_, 34-37 (I962).

Vickery, Brian C. Bradford's Law of Scattering. Journal of Documentation, 4, 198-203 (1948).

Vickery, Brian C. Indicators of the Use of Periodicals. Journal of Librarianship, 1, 170-182 (1969).

Voigt, Melvin J. The Researcher and His Sources of Scientific Information. Libri, % I77-I93 (I959). Voigt, Melvin J. Scientists' Approach to Information. Chicago, ACRL, I96I. (ACRL Monograph No. 24).

Webb, Eugene and J. and Salancik, Jerry R. Notes on the Sociology of Knowledge. Journalism Quarterly, 42, 56I-565 (1965).

Weiss, Paul. Knowledge: A Growth Process. Science, 131, 1716-1719 (I960).

Westbrook, J. H. Identifying Significant Research. Science, 132, 1229-1234 (I960).

Xhignesse, Louis V. and Osgood, Charles E. Bibliographic Citation Characteristics of the Psychological Journal Network in 1950 and in i960. American Psychologist, 22, 778-791 (1967).

Yagi, E. Application of a Type of Matrix to Analyze Cita­ tions of Scientific Papers. American Documentation, 16, 10-19 (1965). 230

VITA

Sheila Joan Kelley Bertram was born March 1, 1940, in

Smooth Rock Falls, Ontario. She attended elementary and secondary schools in Toronto, Ontario, and Kitchener, Ontario, graduating from Eastwood Collegiate Institute, Kitchener,

Ontario, in 1958. That same year she entered McMaster

University, Hamilton, Ontario, and received a B. Sc. in

Honours Chemistry in I962. The following year she attended the Library School, University of Toronto, Toronto, Ontario, receiving a B. L. S. in I963. On September 12, I963, she married Edward Frank Bertram. During I963-I965 she worked as order librarian in the Cameron Library, University of Alberta, Edmonton, Alberta. In I965 she entered the

Graduate School of Library Science, University of Illinois and received a M. S. in I966. Since I966' she has been in residence at the University of Illinois.