Gathering & Evaluating Information From Secondary Sources

Gathering & Evaluating Information from Secondary Sources

Published by the Universities of Edinburgh, Glasgow and Strathclyde

Acknowledgements The material from this booklet has been developed from discussion groups and interviews with the research staff of Glasgow and Strathclyde Universities

The advice and contributions of Mr P.P. Benassi, Dr Avril Davidson, Mr Keri Davies, Prof George Gordon, Prof J.R. Morris, Mr Gerry McBride, Mrs Janice Reid, Dr Alan Taylor and Mrs Sheila Thompson are acknowledged. With special thanks to Miss Alison Faichney of the University of Glasgow library.

The advice of the project Steering Group: Prof Michael Anderson, University of Edinburgh; Dr Nuala Booth, University of Aberdeen; Dr Ian Carter, University of Glasgow; Ms Jean Chandler, University of Glasgow; Dr Avril Davidson, University of Glasgow; Prof George Gordon, University of Strathclyde; Prof Caroline MacDonald, University of Paisley; Prof James McGoldrick, University of Dundee; Dr Alan Runcie, University of Strathclyde; Prof Susan Shaw, University of Strathclyde; Dr Alan Taylor, University of Edinburgh; Prof Rick Trainor, University of Glasgow is also acknowledged.

The project was funded by the Scottish Higher Education Funding Council.

Other titles in Series Gaining Funding for Research Interpreting and Documenting Research and Findings Preparing the Research Brief

© Universities of Edinburgh, Glasgow and Strathclyde 1999 Cartoons D. Brown & W. L. Wilson ISBN 0 85261 687 2 Printed by Universities Design and Print Introduction This booklet is one of a series of booklets aimed at researchers in the early stage of the research career cycle. In this instance the booklet is primarily targeted at postgraduate students beginning their doctoral studies.

The booklet is not intended to be read in one fell swoop, but rather to be dipped into as and when the occasion arises. It is divided into three sections. The first two sections comprise general comments on library and Internet use. These comments are either adapted from library literature, or personal comments from specialists. The third section provides practical advice on the specialised techniques and skill required in the gathering and evaluation of secondary information. This section is subdivided into points of advice, and examples from experienced researchers to highlight these points. At intervals within any section the main points are highlighted by bullet points, the bullet points always refer to the points and examples preceding them. Information for the second section was collected through a series of interviews and discussion groups, which were formed from experienced staff, PhD students, and Contract Research Staff (CRS).

The booklet is not intended to be exhaustive or definitive. The issues raised are those which most exercised the minds of the researchers providing the comments for its preparation. These comments do offer interesting contrasts of opinion, either because commentators disagreed about the way to approach a certain issue, or because researchers from different subjects took different approaches in their methodology. The nature of the examples provided in the booklet are a reflection of the interests of those taking part in the discussions and interviews, and possess no greater significance than that.

There are two appendices, the first gives a list of useful Internet sites, the second a list of useful reference databases.

Whether it is planning future research, putting in grant applications, or placing your results and conclusions in context, the gathering and evaluation of secondary information is very much a basic and essential skill. It is the aspect of research that is first encountered. Be it looking up a word in a dictionary at primary school, or gathering information for an essay at college, we all have some experience upon which we can build.

Three ages in gathering and evaluating ‘by Mr. Shakespeare’

"At first the infant, Mewling and puking in the nurse’s arms.

And then the whining school-boy, with his satchel And shining morning face, creeping like snail Unwillingly to school.

And then the lover, Sighing like furnace, with a woeful ballad"

Contents

SECTION ONE: USING THE LIBRARY...... 5

INTRODUCTION...... 5 PHOTOCOPYING...... 5 ELECTRONIC JOURNALS...... 5 IDENTIFYING USEFUL REFERENCES...... 6 Physical Reference Sources...... 6 Electronic medium...... 7 DATABASE SEARCHING...... 8 Keywords...... 8 Using Truncation...... 8 Using Boolean Operators...... 9 Adjacency...... 9 Field Searching...... 9 Using Parenthesis or Nesting...... 9 Controlled Vocabulary...... 10 Re-using Saved Queries...... 10 Refining Results...... 10 JOURNAL EVALUATION...... 11 Impact Factor (from ISI database information)...... 11 Immediacy Index...... 11 Cited Half-life...... 12 Location...... 12 SECTION TWO: USING THE INTERNET...... 13

INTRODUCTION...... 13 BOOLEAN BASICS...... 13 SAVING DOCUMENTS...... 14 CAPTURE A GRAPHIC...... 14 BOOKMARKING...... 14 SECTION THREE: PRACTICAL APPROACHES TO THE GATHERING AND EVALUATION OF SECONDARY INFORMATION...... 15

INTRODUCTION...... 15 SCOPE OF THE INITIAL INFORMATION GATHERING PROCESS...... 15 Introduction...... 15 Points to Consider...... 15 ENSURING THAT NOTHING IMPORTANT IS MISSED...... 17 Introduction...... 17 Points to Consider...... 17 AWARENESS OF UNPUBLISHED RESEARCH...... 18 Introduction...... 18 Points to Consider...... 18 PROCESSES TO RUN CONCURRENTLY WITH INITIAL INFORMATION GATHERING...... 19 Introduction...... 19 Points to Consider...... 19 DIFFICULTIES IN GATHERING INFORMATION...... 20 Introduction...... 20 Points to Consider...... 20 BEING UNDERSTOOD WHEN REQUESTING INFORMATION...... 22 Introduction...... 22 Points to Consider...... 22 ETHICAL CONSTRAINTS ON THE USE OF SECONDARY DATA...... 23 Introduction...... 23 Points to Consider...... 23 EVALUATING PUBLICATIONS...... 25 Introduction...... 25 Points to Consider...... 25 IDENTIFYING THE RISK OF INACCURATE INFORMATION...... 27 Introduction...... 27 Points to Consider...... 27 BIAS, AND HOW TO AVOID IT...... 31 Introduction...... 31 Points to Consider...... 31 REFERENCES...... 32 Books and other publications...... 32 APPENDIX A - SOME ABSTRACTING AND INDEXING JOURNALS...... 34

INTRODUCTION...... 34 Arts/History/Languages/International...... 34 Economics/Politics/Sociology/Anthropology...... 34 Education...... 34 Engineering/Mathematics...... 35 Law...... 35 Management/Business...... 35 Music...... 35 Theology/Philosophy...... 35 Science/Health (Human and otherwise)...... 36 APPENDIX B - SEARCH ENGINES AND WEB SITES...... 37

BRITISH ACADEMIC RELATED WEBSITES NATIONAL INFORMATION SERVICES AND SYSTEMS...... 37 BUBL INFORMATION SERVICE...... 37 GENERAL WWW SEARCH ENGINES...... 37 Dogpile...... 37 Searching for Softwareg2057...... 38 Subject Gateways...... 38 Evaluation of Information Websites - A Selection...... 38 Section One: Using The Library

Introduction

Meticulous planning and attention to detail are essential in conducting a literature search. The first step is to ensure that you understand fully the services that are offered by the library staff, and what materials the library holds - on the shelves, in reserve collections, on microfiche or microfilm. Bibliographies, abstracts, indices and encyclopaedias can save hours of searching for sources, but they are not always easy to find or to understand (Bell, 1993). However, since 1993 methods of accessing information have changed significantly, and no doubt by 2000 will have shifted substantially again. Details on available information can just as easily be accessed electronically as through the medium of the printed page. Use of library catalogues on the institution’s web site will identify what journals or books are available, and their loan status. BIDS and other databases providing indexing and abstracting facilities are readily available . Support your local librarians- they will help a lot!

The three main sub-sections of Section One provide information on:

 Methods of evaluating journals for their impact within their field.  Techniques for database use,  Methods of identifying useful references,

Photocopying

It is not necessary to photocopy everything. Photocopying is time consuming and expensive. In the long run it is easier to be decisive and restrict your photocopying to articles which are relevant. Not photocopying an article will not lead to future problems as long as you keep your records to a standard which will allow you to return to the article at a later date. If you cannot decide on the importance of an article, try photocopying just the abstract and introduction. Bear in mind that you - and your institution - must make sure copies taken are within the limits of the Copyright Act,1988.

Remember to read the literature you photocopy. It is all too easy to succumb to the temptation of thinking that photocopying is an end in itself!

Electronic Journals

Many journals are now available online, as well as, or instead of, in print. This allows you to consult, print or download (subject to copyright), articles in your own office or laboratory, and can give you faster access and links to, for example, other articles, or databases e.g. Medline citations. There are over 1,000 such journals - and the number grows every day. In most universities, these journals can be accessed from the institution’s networked PCs, and from elsewhere via dial-in services. You should contact the Library Enquiry Service to obtain further information.

Identifying Useful References

Most libraries have literature describing what the library has on offer. This is well worth checking as the variety and detail of this literature can be substantial. Library Internet sites often allow offsite access to catalogues of library contents and loan status of books/journals. One way to start building a good understanding of your local library services is a library induction course, particularly if the institution runs specialist induction courses.

The following section is split into two subsections, identifying references through physical sources, and identifying references through electronic media. At the end of the booklet Appendix A lists a range of electronic and physical reference sources.

Physical Reference Sources

Reference Lists Reference lists at the end of books, journal and review articles are an important method of identifying essential literature. Bear in mind that references in books tend to be about three years out of date by the time they come into print. For more recent references journals are generally better.

Indices and Abstracts There are a wide range of indices and abstracts which allow you to scan many journals at a time, and give you access to resources far greater than that of any single library. These fall broadly into two types. Abstracting journals, organised by subject, which provide, along with the basic information, an abstract giving some details of the paper’s content. Citation indices begin searches with older papers and allow the identification of those contemporary papers which cite the older paper in their reference lists. Abstracting services are only as good as the journals covered, so check that the key publications in your field are included. (Appendix A details some abstracts, databases and citation indexes).

Bibliographies Bibliographies provide a record of what has been published in a particular area. They can be general, or subject specific. International bibliographies (e.g. Répertoire International, for musical sources) can prove valuable sources of reference. It is important to remember that bibliographies can become dated very quickly. If you want to find out what is being published currently, consult major publishers’ catalogues (e.g. Blackwell, Routledge), or, for the most complete record of material published in the U.K., the British National Bibliography (Moore, 1996).

Encyclopaedias and Dictionaries Apart from general basic sources, subject encyclopaedias and dictionaries may be the major reference tools of a discipline. At their best, they can provide authoritative summaries of fundamental principles and current research trends. However, such material can go out of date quickly, and it is a good idea to look at the date of publication, the dates of references it contains, and the general scope of coverage of any reference tool (Moore 1998).

Browsing Consider simply browsing the shelves. Find a book that interests you from a reference list, a verbal recommendation, whatever, and then scan the titles of the books situated on the nearby shelves. Alternately browse the journals. Go to the recent journal section in your library and look through the contents pages of the journals which have some connection to your subject.

Electronic medium

Computer Search Service Most universities offer a Computer Search Service. Among the most comprehensive services, offering access to over 200 databases world-wide via international online hosts, are provided by search analysts on your behalf. Some databases are based on printed sources, e.g. INSPEC, others are available only on computer. They cover a wide range of subjects from Astronomy to Marketing to Zen. They are generally more up-to-date, and offer more sophisticated searching techniques than are available on CD-ROM discs. Make an appointment to discuss your research topic with your specialist librarian. They will advise you as to the most suitable database, and on an appropriate search strategy. (Ashworth, Faichney, 1998)

Reference Manipulation It is possible to create a personal information store, to search by keyword, or to produce lists of references on your subject. It is possible to store your search results in electronic form. They can then be downloaded to your own databases using ‘personal bibliographic database software’, enabling you to keep and search articles based on your own research area. When purchasing ‘personal bibliographic database software’ make certain that it provides software import facilities for the databases which you use regularly.

Updating Services Libraries may offer updating services, which use keywords linked to a personal profile constructed to match your research interests. This profile is regularly run against international databases and the results then sent to you (Faichney, 1998). Libraries may charge for this service, though the cost is usually modest. However, independently of your institution’s library some data sources, e.g. NISS Biomedical, will now send regular updates of references in the subjects of your choosing.

References can be identified by a variety of routes which include:

 reference lists at the end of books/ journal articles  bibliographies  indexes and abstracts  encyclopaedias and dictionaries  browsing  computer search and updating services Database searching

Précis of Marlborough (1998).

The interface (what you see on the computer screen) to databases can vary widely. However, there are particular skills which are common to most, and which should be mastered in order to increase efficient retrieval of information. It can make the difference between finding two papers or 20 papers, or alternatively, reducing a result of 200 papers to a manageable number.

The three basic principles considered in the following section are: choosing keywords; using truncation; using Boolean operators. This section then further examines how these techniques may be used in the location of references.

Keywords It is important when carrying out an electronic search to use the correct terminology. Failure to do so is likely to result either in missing the data sought, or in acquiring far more data than bargained for.

Example: Thus "woodmouse" may bring in references to both A. sylvaticus (found in the British Isles) and P. maniculatus (found in North America). Americans spell labour ‘labor’, so there may be rather limited results if ‘standard’ English spelling is used for searching a North American English language database.

Your librarian should be able to advise you on where guides on terminology are located (e.g. the ‘Synopsis of Animal Classification’, or for anatomical terminology, ‘Nomina Anatomica’) Consider some lateral thinking if searches are producing less than desired: synonyms, animals in a similar ecological niche.

Using Truncation If you enter a keyword in the singular form, you will retrieve references only in that form. As well as singular and plural, many keywords have variant word endings - all of which may be relevant. e.g. computer, computers, computing, crystal, crystals, crystallisation.

In order to reduce typing (never mind hours ‘thinking up’ all possible combinations) most databases allow truncation using a wildcard. This allows the user to type the relevant part of the word stem plus the wildcard symbol.

Examples are comput* , legal*

The * symbol represents the wildcard symbol, which will vary from database to database.

NB. However, if used without due care, wildcards can retrieve rather more than a lifetime’s worth of reading. If this happens, refine your search by adding extra conditions. Using Boolean Operators These operators are used when you wish to specify combinations of keywords during an electronic database search.

AND - "Byzantium" AND "Religion", both words must be present in the reference for retrieval.

OR - "woodmouse" or "long-tailed field mouse" (same animal), if either word is present the reference will be retrieved. This is useful for synonyms, or alternative spellings/topics.

NOT - "weevil not pine", references containing the word pine will not be retrieved.

Thus having found data on the pine weevil you may now wish to find data on other weevil species.

Adjacency It is possible to search for a phrase. In some databases you simply enter the phrase: e.g. lung cancer, heart disease, Russian history.

Alternately the database may need you to indicate that you wish to find one keyword adjacent to another; History (w) Russian, heart (w) disease or, remembering to use truncation, Histor* (w) Russian.

The precise command used to indicate adjacency varies from database to database.

Field Searching Most references in databases are divided into fields. Some examples are ‘author’, ‘title’, ‘document type’.

Example: A record in the BIDS database looks like.

TI: Fly Fishing

AU: Hartley_JR

JN: Yellow Pages, 1981, Vol. 12, No. 10, pp. 806-808

Where TI is title, AU is author, JN is journal details.

In many databases you can look for information in these fields, giving you much more control over your search. Thus it is possible to search for papers written by a particular author, for keywords appearing only in the title field, for a particular journal, for papers published in a particular year or years, written in a particular language, or as a publication type e.g. review.

Using Parenthesis or Nesting It is possible to construct complex search statements. However, to ensure that the search is carried out in the manner desired you must use parenthesis to clarify the logic (in much the same manner as is used in mathematical equations). When in doubt keep it simple and combine it later.

Example: You are looking for recent information on water pollution.

A possible search statement may look like this.

(pollut* or toxic*) and (water or river* or stream* or lake* or reservoir*) and PY³1993

(Again the nature of the commands, * and Publication Year are database specific).

Controlled Vocabulary Many databases use a thesaurus to index records within the database. This is a way of ensuring that all records on a particular topic have the relevant thesaurus term assigned to them. Using thesaurus terms can be a way of ensuring that you get highly relevant information, and reduce the number of ‘false hits’. The thesaurus may refer you to related terms.

Example: The Medline database has a highly structured thesaurus. This would tell you not to use the term ‘cancer’ but to use ‘neoplasm’ and consider using ‘carcinoma’ and ‘tumour’. ‘Cancer’ will produce results, but more relevant papers will be selected using the recommended terms.

Re-using Saved Queries In some databases (e.g. BIOSIS) you can re-use saved queries, showing all of the previous searches you have made with the number of matching entries e.g. Search 1 Author = Auntie Mary articles=1

Search 2 Keyword = canary articles=5000

You can then combine any number of these search results using standard Boolean operators i.e. type in ‘1 AND 2’ will identify articles by Auntie Mary with keyword canary. This is very similar to using operators in the standard search menu but allows you to try different combinations more easily and see how many articles each throws up.

Refining Results One does not always get what one wants, but occasionally what one deserves - usually at the wrong time.

If you get too many papers there are a number of options:

 Check to see if you can use thesaurus keywords  Add additional keywords, or Boolean connectors e.g. NOT  Use more specific keywords  Check to see if you can use thesaurus keywords  Restrict your results by limiting to particular fields e.g. publication years

If you get too few papers, or no papers:  Check that you have no typing or spelling errors within your command  Experiment with truncations  Have you used all possible synonyms or alternative spellings/keywords?  Use the OR command for new combinations  Are there subject or thesaurus terms you could use?

Journal evaluation

It is important to remember when using the journal evaluation measurements listed below that they are not the final and ultimate truth. Journals such as ‘Nature’ are highly cited, but have a definite emphasis in terms of what they will publish. Thus for an ethologist more specialist magazines such as ‘Animal Behaviour’, or ‘Behaviour’, are far more likely to be of interest. Indeed in small specialisms, or in most arts disciplines, there may be few journals which are covered by such measurements. In these circumstances seeking out journals of high impact factor may result in missing valuable information from smaller, less well cited, sources.

Impact Factor (from ISI database information) The Journal impact factor is a measure of the frequency with which the ‘average article’ in a journal has been cited in a particular year. The impact factor will help you evaluate a journal’s relative importance, especially when you compare it to others in the same field. It is calculated by dividing the total number of citations to articles published in the previous two years by the total number of articles published in the two previous years. Obviously there are limitations to the value of impact factor assessments: where delays in publication exceed two years, by definition no paper can be cited within a two year period, also impact factors tend to be dominated by English language, American-cited journals in certain areas of science.

The impact factor is useful in understanding the significance of absolute citation frequencies. It tends to discount the advantage of large journals over small ones, of the frequently issued over the less frequently issued, and older over newer. In each example the first is likely to produce/have produced a larger body of literature than the second. All things being equal, the larger the body of literature, the more the journal will be cited. By providing some qualification of the quantitative data in the Journal Citation Reports (JCR), the impact factor is an important tool for journal evaluation.

Immediacy Index The immediacy index is useful in comparing how quickly journals are cited. Because it is a per article average, the immediacy index tends to discount the advantage of large journals over small ones. However, frequently issued journals may have an advantage here. An article published early in the year has a better chance of being cited than one published later in the year. There is the issue of the length of the waiting list to get published. For comparing journals specialising in cutting edge research, the immediacy index can provide a useful perspective.

The index is calculated by dividing the number of current citations to articles published in the same year by the number of articles published in the current year (i.e. 1979 citations to 1979 articles in journal XXXX, divided by total 1979 articles in journal XXXX).

Cited Half-life The cited half-life is the number of publication years from the current year which account for 50% of current citations received (e.g. in 1988 if the score is 10 then ½ the articles cited are 10 years or older). This figure helps you evaluate the age of the majority of cited articles published in a journal. On the CD-ROM Science and Social Science Citation Index only journals with more than 100 citations are listed with a citation half-life.

Location "Impact factors" etc. are produced by ISI in two forms. Your library may have the information, updated annually, in microfiche, or as a database, called "Journal Citation Reports". This information is rarely available free via the Web, as it is copyright data. (Faichney,1998)

Section Two: Using The Internet

Introduction The Internet is vast in terms of resources and information, finding what you want can be compared to looking for a very small needle in a very large haystack. Thankfully there are search tools available that are just made for the purpose of searching the Internet, these are known as Search Engines (see Appendix B).

There are a number of these search engines, and they are all free to use. To use these engines you need simply to go to their Web Site and enter the text for which you wish to search. Doing this will bring back thousands of results containing one or all the words you entered, many of which may not be relevant to the subject you were looking for. To narrow down your searches you must do what is known as a Boolean Search.

Aside from this brief section there are tutorial sites on how to search the Internet to be found on the Internet, one example of which is http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/

You should also look at subject-related gateways, collections of evaluated sites

(Appendix B).

Boolean Basics To get just what you want you must use the Search Engine’s own Language.

The terms, which are part of a century old system called Boolean Logic, work as effective filters for finding just the information you are looking for. Most of the Search Engines use some form of Boolean query. Check the help sections of the engine you are using to see if it supports Boolean searches.

NEAR - This term will find words located within a certain number of characters of each other. Not every service uses NEAR in the same way, and some do not offer it at all. For example, AltaVista uses it to find words within ten characters of each other, whilst WebCrawler lets you specify the number of characters.

Example for Alta Vista: NEAR finds documents containing both specified words or phrases within 10 words of each other. Mary NEAR lamb would find the nursery rhyme, but probably not religious or Christmas-related documents.

PARENTHESES - Organise your searches even further by using parentheses. NOT cats AND dogs will return pages with dogs in them; NOT (cats AND dogs) will avoid pages with both names.

QUOTES - Putting a string of words into quotes will turn it into a phrase. For example typing "The World Trade Centre" will bring back results with this phrase in them; without the quotes you would have to sift through probably thousands of results containing one or two of the words only. Saving Documents When you have found something useful you may wish to save the document. You can do one of the following, the actual choice will depend upon the software you are using.

1. COPY (click and drag) and PASTE into a document in your own space 2. You can get a copy of the whole document as follows:- In Internet Explorer go to the Navigation icon - to Benchmarks - FILE - then SAVE AS; you should now be able to save this as a document in your own folder 3. In Netscape go to FILE - SAVE AS.

Capture a Graphic

1. In Netscape Navigator point to picture and click right mouse button, you will then be asked for a file name under ‘save image as’. 2. In Internet Explorer the procedure is the same although you now enter a file name under ‘save image as’.

Bookmarking

Having found a useful site, bookmarking makes this quickly obtainable for future reference.

1. In Netscape Navigator click on menu heading BOOKMARKS, 2. Choose ADD BOOKMARK.

It is possible to organise bookmarks for ease of reference.

1. In Netscape Navigator click on menu heading BOOKMARKS, 2. Choose GO TO BOOKMARKS. 3. Choose ITEM, then INSERT FOLDER 4. Name the folder, copy the files to it.

Section Three: Practical Approaches To The Gathering And Evaluation Of Secondary Information

Introduction

The purpose of this section is to provide practical answers to the competencies identified in the Gealy, Westlake & Clarke, (1997) report. The information was gathered from a series of interviews and discussion groups held with university researchers. Participants ranged in experience from postgraduate students to senior members of the research staff. The practical approaches are presented in two forms, a general statement of principles and, where possible, actual examples to highlight the approach suggested.

It may appear at times that some of the advice is contradictory, and indeed it may be so. There are two possible reasons for this. Firstly, the commentators may have been from different academic disciplines, e.g. arts, science, social science. That is not to say that neither could learn from the other, but at times their approaches were divergent. A second possibility was that the two commentators quite simply did not agree and, as there is rarely an absolute right or wrong in these matters, both comments were noted.

On the principle that divergent disciplines can learn from each other, the comments have been grouped by the nature of their aims, rather than by discipline. That some sections are dominated by contributions from a particular field simply reflects the specialisations of those taking part in the project.

Scope of the initial information gathering process

Introduction Faced with a new project there are certain opening moves which should be gone through before committing yourself to a specific line of enquiry. Ensuring that you are thoroughly familiar with the background reading, keeping an open mind at the early stages, talking to and taking advice from your colleagues and supervisor are all vital in building a sound foundation for your future research.

Points to Consider Good supervision is a major factor in assisting a new researcher. New PhD students should make good use of their supervisor to direct their initial reading. Supervisors should help you not only in identifying which are the best opening books, articles to read, but also the electronic aids that are available to help you locate further information.

Make sure that you understand the basics of the subject before you tackle the serious reading. Familiarise yourself with the terminology, techniques and methodologies employed in your new field. Although you may be impatient to delve to the heart of the matter it is important not to skimp on the general background reading. It is not only frustrating to come across words, comments, or references to techniques and theories with which you are unfamiliar, there is the risk that unfamiliarity with key words will result in missing important information during literature searches.

Talk to those familiar with the field, they can help you avoid going down the wrong track and let you know what the current trends are. Similarly, the advice of other researchers can be immensely valuable. They have had recent experience of beginning in a new subject, and can provide helpful pointers to problems for new initiates.

 If you are a PhD student you have a right to expect competent help and support from your supervisor.  Familiarise yourself with the background to the subject before attempting its complexities.  Communicate with your colleagues.

In some subjects it may not be possible to determine the precise scope of the information gathering process at the outset. Indeed the initial step in the information gathering process may be the gathering of information to determine the areas of interest. Start on a broad field, then home in to specific areas of interest. Look at previous work to see if it has concentrated on one aspect of the subject more than another. Ask yourself why they have concentrated on these areas. Is it because they have not had access to the primary sources, is there a genuine gap in previous studies in this field?

In the arts it may be advantageous to begin reading the secondary material, rather than the source material itself. Collected essays, when they are from a range of authors, are good for identifying where the major current debates lie in relation to the interpretation of primary material. Use of collected essays not only allows identification of where the debates are, but also where the gaps in the information are. Ask yourself the question, are the topics I think important being discussed already, if not, why not?

Example: An American edition of ‘Wuthering Heights’ was produced which contained five essays on the book, each by a radically different critic. This style of book has the advantage of allowing an immediate comparison of a range of interpretations. Another useful aspect of this book was the placing of the footnotes relating to the different approaches in the same volume as the primary text.

On the principle that recent secondary material both refers to and covers the earlier comments, the initial reading process may, in some subjects, be considered complete after most of the material published in the last 20 to 25 years had been gathered. However, although the material of the last 20-25 years may cover all the major contributions to that field, there is a risk that any idea unfashionable over that period will be missed if an examination of earlier texts is completely excluded.

 Keep an open mind at the beginning of a new project.  Try to use your initial reading to identify interesting areas of study.  Consider reading the secondary material before source material.  Collected essays (subject reviews) are better than monographs for identifying areas of debate and gaps in knowledge.  Finally, remain flexible in your data gathering, projects can change in emphasis as they proceed, avoid allowing your thinking and data collection to sink into a rut.

Ensuring that nothing important is missed

Introduction Communication with colleagues, and a thorough use of review papers and electronic databases are vital to ensure that no major papers are missed. But researchers should take care to ensure that information gathering is an ongoing process to avoid finding that their publications have omitted a major paper published shortly before the end of their project.

Points to Consider It is essential to check reference lists at the end of recent publications. Reviews can be particularly useful in locating hitherto unnoticed references.

Talk to people, not only those within your own field, but also those outwith. Fields of research tend to overlap, historians may have interesting comments on art, a geneticist may have a good reference for a population ecologist. Conferences are a good source, and not just the presentations themselves. One of the most valuable aspects of conferencing is the opportunity to chat informally to others in your field. For new researchers it is worth remembering that other researchers and staff can benefit from your reading lists to help keep themselves up to date with the literature, do not imagine that you have nothing to contribute.

Ideas go in and out of fashion, do not rely entirely on recent publications. Check a few of the classical texts, these may provide references to ideas not currently in vogue.

Check online databases to ensure that no references have been missed. These checks should involve at least two separate databases (e.g. for medics Embase and MEDLINE), no one database will possess all the references. Local experts, including library staff, can provide considerable help in literature searches, particularly for the inexperienced.

Try to identify key authors, e.g. those that most of the major texts cite, their use when searching citation indexes can significantly reduce the chance of missing relevant material.

As you proceed take the opportunity to present your work in departmental/professional seminars and conferences. The feedback gained from these ‘adventures’ is usually instructive, not only during the question period following your presentation but in informal discussion afterwards. At the same time take the opportunity to collect the e-mail addresses of people in your field. The regular exchange of information and ideas is another good way of ensuring you miss nothing. Though some care might be taken to ensure that you do not give away new ideas before you have had a chance to develop and seek funding for them. Perhaps one of the most likely times for gaps to appear is before the final completion of the final project. It may have been some years since the initial literature checks, a final run through the literature is vital at this point.

 Present your work at seminars and conferences, the feedback can be invaluable.  Use at least two databases, no database will have all the references.  Check old references, ideas go in and out of vogue.  Talk to people, and not just those immediately involved in your own field.  Remember that last literature review before completion.  Make full use of the reference lists in journals and books.

Awareness of unpublished research

Introduction Communication with your colleagues in the field is a recurring theme throughout this booklet, and an awareness of ongoing work is absolutely vital. Potentially a lot of the information on who is funding what, and who is doing what, can be located on the Internet.

Points to Consider Talk to others in the field. It may be possible to use the Internet to keep track of new work. However, with increasing competition between research groups, and the inevitable parallel increase in secrecy, it is unlikely that anything on the Net will be genuinely leading edge. Generally the web tends to detail papers near completion or completed and awaiting publication.

The majority of grant funding agencies publicise the grants that they have awarded. It is possible to write to the grant funding bodies (especially the charities), and they will provide information on the areas which they are funding. The agencies will often supply a brief abstract that refers to novel findings, or a new hypothesis. The funding agencies supply (at least annually) reports of their funding patterns which delineate areas of funding, and include reports from laboratories on research grants awarded and work undertaken (the latter may include some data). These reports can be useful in allowing an awareness of developments in other research laboratories with which you are not in direct contact. This is helpful because it allows you to target resources towards areas that are currently being funded, or to stay away because the area is ‘over-subscribed’. More importantly, some of the grants may be very speculative, and the abstract could allow you to understand where a body of work may be heading over the next three to five years, rather than where it was six months ago as with published work.

Conferences, or the informal contacts made at them, are essential in this matter. But make sure that you use new contacts to the full, keep in touch after the event, write to them to find out what they are doing. Informal coffee room chats can often let you know if you are not up to date.

Example: The postgraduates of one university were described as musk oxen. They would come into the social gatherings at conference, then stand in a circle speaking only to those that they already knew (basically themselves), making no new contacts whatsoever. It may take a lot of courage, especially if your companions are musk oxen, to make new contacts, but the rewards for making the effort can be very significant.

At conference making a poster or oral presentation enhances the likelihood of meeting other workers in the field and obtaining information from them.

 Use the Internet, but do not rely on it for information on unpublished work, especially work/ideas in early stages.  Keep an eye on to whom, and for what, the funding bodies are providing cash.  Network (relax), do not be a musk ox.

Processes to run concurrently with initial information gathering

Introduction For new researchers this is a vital aspect of information gathering, and one which must be got to grips with as soon as possible. It is easy for inexperienced researchers, who are as yet unaware of how ridiculously long some processes take, to fall into the trap of concentrating on early reading to the neglect of all other activities. The advice below list some actions which you would be well advised to begin whilst your reading progresses.

Points to Consider The most significant processes to begin early are those reliant upon a response from others. For a field biologist or social scientist this may be the gathering of volunteers to help in surveys, for a laboratory based study the availability of essential materials should be confirmed long before experimentation is due to begin. Ask yourself a few questions. Is a reagent available commercially, can it be produced in-house? Write to potential donors of materials (plasmids, wooden billets for weevil traps, copies of letters). Remember to check what the price is; financial, acknowledgement, co- authorship. You need to expect some problems regardless - so always check your source of supply. Be aware that there may be contractual obligations in receiving material, which will usually need to be cleared through your institution’s Research or Industrial Liaison Officer.

Similarly it is important to check methodologies during the initial reading period, what is and what is not possible. Although methodologies may be referred to in the literature, a precise description can be hard to come by, authors often omitting steps which appear obvious to themselves, but which can be less than obvious to the uninitiated.

Example: This case, from a PhD study radio-tracking woodmice, highlights some of the above points. Whilst there were several descriptions of techniques, securing a radio-collar onto a mouse proved more difficult than anticipated. The woodmice proved generally unenthusiastic. Holding the mouse, tightening the butterfly antenna using small pliers, attempting to heat shrink a rubber band to secure the radio-collar, whilst taking care not to burn mouse or self with the miniature soldering iron, proved near impossible. Eventually, on speaking to the authors of the earlier publications the researcher discovered that the papers described two very different methodologies. The method the researcher was using had a superior lightweight collar, but required a lightly sedated mouse. The second paper used a heavier collar, but did not require sedation of the mouse.

Are you a competent statistician, or are you going to make use of a statistician’s services? If the latter is the case it is worth while speaking to your advisor before beginning the work. Statisticians very definitely like to advise people on data analysis before the data have been collected.

Essential data gathering from other bodies often requires an early start. Not all government agencies and voluntary bodies have their data on computer files; they might just as easily be in a series of boxes in the attic. If this is the case, and the data are needed early in the project, it is essential to begin making the arrangements as soon as possible.

Example: When a PhD student first sought information from the Government agency on the distribution of mayfly in streams in North East Scotland they proved uncooperative. Eventually, having obtained the agreement necessary to be allowed access to the Agency database, he found that the data were not computer based records but instead on thousands of data sheets stored in box files.

 Reading can become tedious and its value decline; avoid making it your sole occupation for months on end.  Do not wait until the reading is completed before contacting others for help/information.  A good way to ensure a problem is to take the word of others that there is no problem - check it out.  Contact statisticians in the early stages for advice!  Start to master the essential methodologies in parallel with the initial reading.  Not everybody has discovered computers, or has the funds to make full use of them, you may need time to collect data.  Not all data will be of expected value, whatever people lead you to believe they have, do not rely on it to replace your own data collection process.  Finally, if the information needed is identified as difficult to gather in the section below - start gathering now!

Difficulties in gathering information

Introduction The difficulties which may arise range across a wide spectrum. The mobility of a large section of the academic community can make tracing a specific individual very difficult, some individuals and organisations may simply refuse to help. Imagination, flexibility and forward planning will greatly reduce the number and level of problems encountered.

Points to Consider The response to requests for information and help varies from individual to individual and organisation to organisation. If you need assistance from others, ask colleagues in the department if they have collaborated with the organisation in the past. It may be that success/speed of response is heavily dependent upon the person within the agency that you contact. It is not always easy to track down authors of papers, they can prove a mobile collection of individuals with a habit of moving between research institutions. Personal contacts can be useful in helping you to track down individuals. The contacts need not necessarily be people you know, new contacts can be made as the occasion requires. Perseverance is necessary.

Example: A recent publication by the researcher will provide the postal address of the host institution. International directory enquiries should be able to provide the telephone or fax number for that institution. An Internet search on the institution may allow access to the staff directory, and from that the current e-mail address for the person in question or alternately, the head of department who may know where the person has gone.

Do not assume that the individual will be willing to communicate in English. In order to avoid giving offence, and in the hope that due consideration might increase the probability of co-operation, some researchers append a comment to the effect that, "I am writing in English because I do not speak your language, but hope that as your publication was in English, this will be acceptable".

After tracking the person down there is no guarantee of co-operation, even if assistance has been given to other researchers and groups in the past (see ‘ethical constraints’).

Projects in new fields can have particular difficulties in locating information. You may know, or at least hope, that the information is there, but if previously tried and tested methods are non-existent, or of only limited availability, then locating the information requires persistence and flexibility.

Example: One of the current research programmes of a contributor considered appetite regulation in juvenile salmon. The project was examining the newly- discovered appetite/fat reserve-controlling hormone, leptin, in fish. In order to do this a salmon "cDNA library" (a complete set of DNA sequences) would have been very helpful. However, the researcher was unfamiliar with this branch of the science and was unaware of whether such a library existed. Several personal contacts were e-mailed to ask if they knew of the existence of such a library, or of any other researchers who may have some information on the subject. Of the four or five people contacted initially, none had any first-hand information, but three suggested other contacts. One of these suggested a researcher in Newfoundland, who had some salmon cDNA libraries, and who expressed a keen interest in the proposed work - a potential future collaborator.

 Think laterally.  Use personal contacts to build links to other research groups and sources of information.  Persistence pays off.  Regard the collection and interpretation of data as mutually complementary processes.  Explore a range of sources simultaneously, they may interact to give new insight.  Be imaginative and flexible.  Some government agencies are not renowned for their speed of response: identify your point of contact carefully.  Co-operation is not guaranteed, try to have more than one iron in the fire.  Start now, it may take some time to find the people you want.  Consider acknowledging your failure to write in the native language of the person from whom you are seeking assistance.  There are a range of techniques available for tracking down potential sources of advice, these include; old publications, e-mail/letter enquiries to colleagues, Internet, International directory enquiries.

Being understood when requesting information

Introduction Being understood is about getting the information you want, and not what somebody else thought you wanted. Generally verbal communication is not ideal without a written back up, in which, to avoid confusion, jargon should be kept to a minimum.

Points to Consider The best way of being certain that a request for information is being understood is by ensuring that your terms of reference are common. This is most likely when the person with whom you are speaking has a good theoretical background in your field, and when you limit the jargon used.

Example: For a student of English literature studying the role of women in 17th century novels/plays it may be important to set your comments within the historical context of the role of women in the 17th century. Whilst it is likely that a 17th century historian specialising in feminist issues will appreciate your terms of reference, it is unlikely that an historian of the same period, but working in another discipline, will not.

Do not make a single request for the information: if there is any likelihood of being misunderstood repeat the question, discuss it verbally, but also send the question in written form. It may be that you feel the verbal request was clear, and that you both understood it. But if the person on the other side of the line is not dealing with your request for several days, or in their native language, then memory and clarity may both fade.

One of the PhD students in the discussion group drew a line between approaching familiar contacts, with whom students may be comfortable, and unfamiliar and/or senior members of staff, with whom conversations may be intimidating. In the latter circumstances a written, rather than a verbal request, would help to ensure that there is no confusion. A written request will also reduce the risk of the researcher being given the information that the ‘wiser’ head thought would be useful, as against what was actually wanted.

If you need a constant supply of information/assistance from a source, keep them informed of your activities. This will help those providing the information to understand what you want from them. In the circumstances described below the researcher benefited considerably from regularly updating collaborators on his progress. In this case by reassuring them that their concerns were unjustified. (Before deciding to forward information, take advice from colleagues).

Example: One project, which formed part of a PhD study, required two fixed sample lines to be located on a private reserve. Water level range gauges and dipwells were located along the length of the sample line to provide essential hydrological and hydrochemical data. The reserve warden was concerned as to the possible visual impact of the equipment on the reserve. After completing a first year report, which included a detailed description of both the work in progress, and planned future work, a copy was sent to the site warden. Being given the opportunity to read the report allowed the warden greater opportunity to understand the relevance to himself of the project’s aims, which in turn made negotiating site access easier.

 Make certain you are using the same terms of reference.  Do not rely upon verbal communication, send a written request - especially if you are uncomfortable/nervous dealing with the person providing the information.  Make certain the information you get is the information you want.  If you need assistance from an organisation/individuals, keep them informed of your progress and future plans. They may not read the information you send them, but they will appreciate the consideration.

Ethical constraints on the use of secondary data

Introduction Most institutions and professional bodies will possess a code of ethics; researchers should ensure that they are fully aware of this code. The following section considers the rights of the providers of information, the original subjects from which the data were obtained and the relationship between yourself and your colleagues.

Points to Consider If information from another source has been published then it must be acknowledged. But not only acknowledged. It is important to ensure that the intended use of the data is acceptable to the provider. Failure to ensure that the use of the data is acceptable to the provider may not only result in their refusal to collaborate with you in the future, but may also damage the research projects of others.

Example: On requesting information on the distribution of lochs containing a particular freshwater fish from the foremost authority on this subject, the researcher was denied this information. This was despite both the researcher and the person holding the information having the same conservation objectives. The reason given was that this information had previously been provided to an outside body which had then used the information in a manner contrary to the wishes of the provider.

Not only should the proposed use of the data be acceptable to the provider, any guarantees given to the original subjects must be borne in mind.

The need to check for permission to cite information may be partly gauged by the environment in which the information was provided. In a formal presentation - at conference. - it is unlikely that the author will have any concerns with comments being cited. A casual remark is, however, a very different affair. Correspondents will definitely be aggrieved if you publish their work before they do.

Example: Whilst working in Ireland an experienced researcher showed a colleague some early drafts of a paper on which she was working. A few months after moving to Scotland she received a copy of a research proposal from her former colleague. This proposal had taken her work and presented it as an original idea, with no acknowledgement of her contribution. The proposal plagiarised, word for word, entire paragraphs of the researcher’s draft manuscript. Further confirmation that the researcher’s draft manuscript was the source of the proposal came from references to Greek publications, a language which the researcher, but not the colleague, could read.

Plagiarism as blatant as this is generally unusual, but is a clear example of unethical behaviour. The best way of protecting yourself from such behaviour is to listen to the advice of your colleagues as to who can, or cannot, be trusted. However, do not allow the presence of a few rogues to discourage you from testing ideas out on colleagues.

 Check that the information is being used in a manner acceptable to the provider.  Check that your use of the information does not breach the original conditions of collection.  Ask yourself, would I provide this information, if not, perhaps you are asking too much.  Acknowledge sources of information.  Confirm that your use of the information is acceptable to your source.

Medical consent forms are not always rigorous, and are not infrequently written to ensure the widest possible legal interpretation. Researchers must bear in mind that legal and ethical are not one and the same. What is legal, is not necessarily ethical and vice versa.

Where volunteers are being used to collect data, the researcher has a moral duty to ensure that they are acting according to the correct ethical standards. It is not an excuse to claim that you did not know of unethical behaviour in people working for you. You should know, and should have made it clear beforehand what was/was not acceptable.

The following example from the experiences of Stephen Waters (Bell, 1993) provides interesting insights into the difficulties of carrying out research on one’s own colleagues. Although the example is predominately one of gathering the basic data, nonetheless, there are points to be learnt regarding what happens to that data after it become secondary published data.

Example: Stephen Waters was a teacher who decided, as part of an Open University course, to investigate the role of his own head of department. He went through a fairly prolonged negotiation period to reassure colleagues as to his trustworthiness before embarking on his programme of research. Of interest here are some of the comments he later made about the guarantees he had given prior to undertaking the research. He had promised all participants an opportunity to verify statements prior to production of the final report. This proved almost impossible, most participants not having time to read the entire manuscript. Therefore, lacking time to identify all their comments within the manuscript. He had further promised all participants a copy of the final report, which ultimately cost rather more than anticipated. The ethical agreement he reached with contributors was only made verbally. This created problems at a later stage, when it transpired that none of the contributors could precisely recall the conditions agreed upon. In retrospect he regretted not providing them with a written copy of the agreement. However, it was in seeking to publish the data that the greatest problems arose. All contributors had been promised anonymity. A promise which could be met externally, nobody outwith the school could identify the contributors. However, as all the contributors came from the same school, it proved impossible to provide anonymity internally.

 Remember that legal and ethical are not interchangeable terms.  If you use others to collect data, you have a responsibility to ensure their ethical behaviour.  Even with your colleagues, you must earn their trust.  Consider costs (e.g. time and/or money) before making promises to contributors.  If guarantees are required, and they are of any length, record the agreement in writing.  Where guarantees of anonymity are provided they must be valid under all circumstances, unless otherwise agreed.

Your sins will find you out, a cautionary tale from the world of biology.

Example: A colleague of the researcher was having trouble cutting a plasmid given to her by a visiting worker; eventually she contacted the researcher who had generated the original plasmid only to discover that he was not aware that the plasmid had left his laboratory. The visiting worker had ‘acquired’ some of it whilst working in the other researcher’s institute! This sort of thing is totally unnecessary and entirely dishonest - most scientists are very generous and will happily give out their plasmids/vectors.

Evaluating publications

Introduction Do you agree with the argument, if not why not? Even if you can find no point of agreement with the publication, does it advance your argument? How valid are the conclusions or parallels drawn, is the basis well founded, or are the claims of the authors specious? This section considers where less reliable information is most likely to be found, and suggests various methods by which publications may be evaluated.

Points to Consider Ask yourself if you agree with the conclusions in the paper, and if not, why not?

Having decided on agreement, or disagreement, what is the value of the paper to your own work, how will using the material help advance your argument? Even if you do not agree with the overall thrust of the author’s argument this does not necessarily reduce the value of the publication to your research. The material can still be useful in providing historical or other information, regardless of interpretation.

One method of estimating the possible importance of a paper to your work is to break the paper down into a series of fields/categories. For example: species used, branch of subject, period of art, location of study (laboratory, field, continent). These categories are often similar to the key words used in the paper. Each can then be further sub-divided into increasingly specialised areas. The further down this hierarchy the subject matter of the paper proceeds towards one’s own interests, the more useful it is likely to be.

One topic which can be particularly difficult when starting out is the evaluation of schools of thought. You might be attracted to the work of a certain group. How do you evaluate their material in a way that allows you to gain a fuller understanding of their research criteria? One way to do so is to use the suggestions made in the example below to check the claims being made by the group whose material is being analysed. Other possibilities include examining collections of essays by the same group; that way you can usually find out what their primary concerns are, especially if the collection includes a coherent introduction. Tertiary material is important here, i.e. reviews of the work of such groups: writing about the writing, rather than the primary material. The final method is conversation with others working in the same area as yourself: your supervisor, other research students, people from other institutions. As your confidence and experience grows it will become easier to reach a definite conclusion about how your work relates to other research groups.

Example: Several influential collections of essays were published on Shakespeare in the mid-1980s. One of which took, as its starting point, the structures of power in the English American colonies, then applied the conclusions drawn from the American situation to Shakespeare’s London, especially in the performance of politically sensitive plays. The dramas used are Shakespeare’s historical plays covering the career of Henry V, both before and after his succession to the throne. The essay is famous both for its stylistic brilliance, and the issues of historiography that it raised. This essay is usually quoted as primary text for the group of critics known as the American New Historicists. However, its conclusions can easily be challenged. Firstly, the fledgling American colonies were ruled by military governors, and were subject to a form of martial law. To translate the implications of this form of government wholesale to contemporary London is, at best, misleading. Secondly, there are only three paragraphs in a 27-page essay on the Henry V plays, yet these plays were, ostensibly, the main concern of the essay. Thirdly, the essay gives no indication of how the Henry plays rework historical material that was already two centuries old by the time Shakespeare took up his quill. The mature critique of this essay would question its historical veracity: does the New Historicist historicise at all?

Similarly do not confuse interesting with useful or informative. One researcher admitted that she had recently published the most boring material she had ever read. But argued that although the paper, defining genetic code, was of a singularly monotonous nature, nonetheless, the paper presented considerable potential as the basis for future work. There are times when you will feel it is necessary to reject the conclusions drawn in a publication. However, this is not necessarily a reason to reject the paper outright. The spin that people put on the discussion may not fit in with your idea, but at least the data are there. It may be possible to use the data/results by commenting upon alternate interpretations.

 Do you agree with the argument, if not, why not?  How does the publication advance your own argument?  Consider how close the material lies to your own field of research.  Use reviews of the work in question to help yourself generate ideas and evaluate the publication.  Find other work by the same school to identify their main areas of concern.  Consider the validity of the parallels drawn - realistic, or merely convenient.  How much attention is paid to the primary source, this can be lost in a welter of secondary argument.  That the paper is a cure for insomnia does not mean it contains nothing of intrinsic value.  Rejecting the conclusions of an author does not mean that the publication is of no value.

Identifying the risk of inaccurate information

Introduction It is important to consider how the information was presented, non-refereed material inevitably carries greater risk of inaccuracy. The contributors considered inaccuracy to be more likely when the work of an individual crossed interdisciplinary boundaries. The final point is a warning against accepting material at face value.

Points to Consider Do not assume that information presented informally, or at conference, was necessarily accurate. Enthusiasm can carry researchers along, encouraging them to leave checks and balances to a later stage. This can necessitate a substantial revision of conclusions, or results, after the checks have been introduced. Similarly anecdotal information, from amateur or other local sources, can be immensely useful - or immensely misleading. It is often difficult to judge which. Generally it is wise to check this information out with colleagues or collaborative sources.

Example: A PhD student found it necessary to conduct an experiment examining the potential of the ammonium ion as an activator of the enzyme sodium-potassium ATPase in the fish gill. The first time the experiment was performed the result was positive. An exciting result, which the student quickly passed on to the supervisor. However, the necessary sample number for reliable statistical analysis required the experiment to be run several times. On each, and every subsequent occasion, the results were negative, the initial positive result had to be discounted. Unfortunately, by this time the supervisor had told a (fortunately small) number of colleagues about the false positive result, which they then had to retract.

Be especially careful of conclusions drawn in publications which cross inter-disciplinary boundaries. If in doubt check with colleagues in the relevant fields, or expand your reading to give yourself a greater insight into those fields with which you are relatively unacquainted.

Example: A few years ago a very well known and respected writer published a major book on Renaissance culture. The author investigated a whole host of interdisciplinary concerns, which ranged through history, literature, and art. However, in one example she was wrong - at least, according to specialist art historians. She took a line on Renaissance iconography that borrowed heavily from the classics, especially in relation to the exploitation of women. Her example was the painting of the Rape of the Sabine Women. Despite its title, the painting in fact depicts the Sabine women reconciling the Sabines and the Romans. The painting examined only one incident in the story. The critic was wrong: rather than showing women being raped, the painting shows them acting as mediators. This example is especially useful, since the critic was trying to integrate art history, literary studies, and the classics; her project fails because of an error in an area outside her own immediate specialism.

 Work which crosses inter-disciplinary boundaries can be especially prone to errors.  Conference or seminar presentations should not be regarded as the new gospel.  Consider the reputation of the source, but do not be put off by controversy.

An understanding of basic statistical principles is vital to checking the validity of analysis. Errors in statistical analysis are surprisingly commonplace, and new errors can go unspotted for considerable lengths of time.

Example: Population studies that examined trends in populations over time frequently repeat the error of temporal or spatial pseudo-replication. These studies take the population sizes measured at the same site over a period of months as independent data, whereas they are repeated measures.

A fair reliance but not absolute can be put on referreed papers. Even in referreed papers there are a few pointers of which to be aware. Look for a lack of proper explanation in the data, e.g. a band in the photograph of the gel which has not been properly explained. Cross-reference data with earlier publications, or look for inappropriate phrasing. It was widely agreed that one of the best indications of a dubious paper was an over-interpretation of the results, often indicated by highly speculative discussions.

It is important to examine the results. What are the technical limitations of the study? Check that the claims made in the conclusion do not exceed these limitations.

Example: A simple example of inadequate methodology invalidating conclusions may be found in a study using a fluorescent calcium indicator to follow changes in intracellular calcium. All fluorescent probes have a limited range over which they can follow the intracellular changes in calcium accurately. This study used an indicator that did not accurately follow relatively low calcium levels, and then suggested that a significant alteration in low levels of calcium had been caused by the experimental manipulation. The data were flawed, in this instance because of the inadequacy of the probe to reproduce events at low calcium levels.

A parallel between the examination of results and their statistical analysis, may be drawn with the suggestion that, in the arts, a flaw in the ‘data’ may be used to indicate dubiety in the conclusions. Sometimes even slight inaccuracies can cause real trouble, especially if they are related to the major research claims.

Example: An influential book on the inter-relationship between poetry and politics in the 17th century was published in the early Eighties. However, the book was marred by a basic historical error. The writer notes in passing that John Donne was Bishop of London. He was in fact Dean of St Paul’s. This error may appear unimportant, but does indicate that further checks may be wise.

The extent to which the argument stands up without the reader having to refer to the footnotes or to other sources of supporting evidence can be a helpful initial pointer to the strength of a secondary source. However, art history which does not show evidence in its footnotes of having taken into account relevant original works of art, circumstantial evidence, and written primary sources (e.g. period letters, reviews or other texts), or which depends on citing a few which are readily available in common secondary sources, may be suspect if it claims to offer a ‘new’ reading or interpretation of work(s) of art/ideas. If the footnotes show that an author has made use of archive, collaboration or other primary sources not hitherto or commonly used in relation to the subject, this may lend credibility to claimed ‘new’ conclusions. Though, if the footnotes reveal too exclusive or selective a range of primary sources, however ‘original’ these may be, this indicates a need for caution.

There are exceptions to the ‘primary sources’ test, e.g. a work of mediaeval art for which little comparable art may survive, and there may be no relevant period texts. In this case the discussion should be able to draw on circumstantial evidence. Circumstantial evidence may include the location where the object was found, and its possible purpose as indicated by size, shape, ornamentation.

Evidence of reference to relevant primary, or circumstantial sources, is not a sufficient test if the argument constructed has gaps, or contradicts itself, and makes no effort to account for these.

Close attention to primary material is absolutely necessary when considering major claims to originality. When dealing with secondary material which makes such claims it is always wise to assume a posture of extreme scepticism.

Two Examples: (i) In the 19th century the Scottish painter William Dyce painted the ‘Highland Ferryman’. This painting was assumed, in the standard secondary literature, to have been painted from nature when Dyce was in Scotland in 1857 and living on Deeside. However, the ferryman and surrounding scenery were out of scale, indicating that the painting may have been a composite of a photograph and a model in the artist’s studio. The researcher then sought evidence of the photograph. But the evidence uncovered contradicted both earlier hypotheses, indicating that the painting was neither a real life scene, nor a composite. George Washington Wilson (a photographer of the period) described a photograph he had taken which exactly matched the painting, and which indeed, Wilson claimed was the basis for the painting. The conundrum was eventually solved by looking at the problem in reverse. In this case examining the source not of the painting, but the photograph. This was the origin of the composite, a photograph created from two or more negatives. (Willsdon, 1990)

(ii) One student once wrote in considerable detail about the violet colour scheme in a Monet painting, not realising that the violet was an artefact of a second rate photograph.

 Check that all available evidence was properly considered, that the argument presented was not overly selective of the data.  Check the data upon which the publication was based were valid.  Check that the statistical analysis upon which the publication was based was valid.  Confirm that the methodology and data could provide the results claimed.  Check secondary material against the primary source.

Current ideas and knowledge are not the only useful sources of information In the example below another researcher used the knowledge base and views of an earlier period because they offered a more relevant guide than a modern source to the interpretation of painting completed in 1900. In this instance the researcher sought out, not what secondary material was now available, but what secondary material had been available at the turn of the century.

Example: Although referring to recent published history may give a summation of research to date on the signing of the Magna Carta, it may not indicate how much of this information would have been available to the artist who portrayed the subject at the Royal Exchange in 1900. The researcher consulted history textbooks in use at the turn of the century to identify the views and knowledge which were current at that period with regard to the signing of the Magna Carta. These revealed that there had been, at that time, considerable interest in this event as a key foundation, not just for the modern parliamentary system, but in particular for London’s historic system of government. In combination with other contextual evidence, this finding led to the conclusion that the subject composition and imagery, of this, and other Exchange murals, were intended as ‘propaganda’ for the preservation of London’s traditional government in the face of threats to its survival from the new London County Council (Willsdon, 1997).

Consider reversing the usual conclusions and looking at the material ‘back to front’.

Example: In this case the researcher wanted to consider the perspective of the people for whom the mural, Gustav Klimt’s Beethoven Frieze (1902) had been produced, rather than her own late 20th century perspective. This arose from a feeling that the ‘standard’ interpretation of Klimt’s frieze, as an illustration of words from Schiller’s ‘Ode to Joy’, was too simplistic. The standard explanation did not allow for the frieze’s wider cultural context, turn of the century Vienna. The researcher therefore explored the extreme (null) hypothesis, that the frieze had little, or only limited, connection with Schiller’s ‘Ode’. The researcher looked at the frieze not from a literal standpoint, which had assumed that because part of it had been exhibited by Klimt with a label quoting Schiller’s ‘Ode’, it must illustrate the ‘Ode’. Rather the researcher considered the frieze from the much larger perspective of Klimt and his contemporaries, who were familiar with a complex framework of literary, musical and philosophical reference no longer shared by modern viewers (Willsdon, 1996).

All contributors were unanimous in their distrust of unrefereed material on the Internet. Do not be put off an idea merely because it is controversial, but exercise extra caution when dealing with unrefereed papers.

Another way of ensuring that conclusions are within a reasonable margin of error is to utilise a recognised standard, or set of standards. In arts- based subjects this can be difficult, but there are possibilities. Use of an internationally accepted standard, such as the ‘Oxford English Dictionary’, can be an invaluable tool in deciding how acceptable the conclusions might be. This kind of approach can lead to a whole series of analyses which are verifiable, and which may even challenge accepted thinking.

Example: There is a famous line in Shakespeare’s Sonnet 18, which includes the phrase "the darling buds of May". Conventionally, this has been taken to be a nice, sweet reference to the kinds of things that Shakespeare is assumed to mean: the nice English countryside, romantic love (e.g. A Midsummer Night’s Dream). However, a simple look at the Oxford English Dictionary shows that "bud" is Renaissance slang for breast, and that "darling" was a kind of apple sold on the streets of London in 1598 - a Renaissance Granny Smith’s. Breasts have obvious sexual connotations, as does the apple, the ultimate sign of female sexuality and duplicity in western patriarchal religious discourse: Eve in the Garden of Eden. "May" refers simultaneously to the Virgin Mary and the month of May, conventionally associated in Renaissance iconography with country matters, especially sexuality. The inescapable conclusion is that the meanings of modern sweetness are unacceptable as the only possibility if one wishes to place these poems in their historical context.

 Check that interpretations are relevant to the context of the times.  Scepticism should be adopted at all times.  Received wisdom is no guarantee of accuracy, consider alternative interpretations of the data.  Check statements against internationally recognised standards.

Bias, and how to avoid it

Introduction There can be no question that we all bring some degree of bias to whatever we read. The issue is less can we avoid bias as how to recognise it and ameliorate its effects. The following section provides a few suggestions.

Points to Consider It is difficult to avoid this problem, perhaps impossible, was the comment of several optimists on the discussion groups. It was suggested that the cultivation of extreme scepticism, even towards publications with which you agree, is a step toward the goal of eradicating bias. This is difficult to do, but once you begin to question the literature that once you agreed with, then you stand upon the threshold.

Try turning your research idea around. If you believe that your methodology produces the correct results take the opposite point of view. Consider the world from the new perspective. My methodology cannot produce valid results, only this alternative technique can.

Try to be inclined towards work which does not fit your favoured world view, attempt to counterbalance by trying to adopt a positive approach to papers which run contrary to your expectations.

If data is contrary to your own, approach the laboratory in question and ask to swap antibodies or material with them to re-run your/their experiments, but not all institutes will agree to this procedure.

Finally give the publication some thought at a later date. There are few of us who can listen to an argument or read a paper then alter our ‘world view’. Most of us can alter our views on reflection. If there is an author with whom you strongly disagree, try going back to him/her in a few weeks time.

World views change, that which is acceptable today may well be ridiculed tomorrow, try not to let convention determine your outlook.

Example: In botany, circa 1850, the general morphology of the root was described as having three layers to the vertical structure. This was repeated in text books until 1950, when someone announced that he could not actually see this structure down the microscope. This picture had been drawn, redrawn and photographed for around a hundred years.

Remember Darwin/Wallace, Mendel or Heinrich Schliemann: in each case their work was controversial, forgotten, or considered somewhat eccentric.

 Develop healthy scepticism, try to get into the habit of questioning even that material with which you agree.  Consider the opposite view: my way cannot work, but theirs can.  Can you swap material with those you disagree with?  Give it time, think it over.  Remember yesterday’s eccentric/heretic may be tomorrow’s hero.

References

Books and other publications

Bell, J. (1993) Doing Your Research Project: A Guide for First-Time Researchers in Education and Social Science. Open University Press, Buckingham. 176 pp.

Brown, S., McDowell, E., Race, P. (1995) 500 Tips for Research Students. Kogan Page Ltd, London. 127 pp. Gealy, N. and Clarke, D. (1998) Development of an Interim Workplan for the Researcher’s Lead Body. Maloney and Gealy, 24-26 Mossbury Rd. London. 30 pp.

Gealy, N., Westlake, D., & Clarke, D. (1997) Draft Occupational Standards In Research. Maloney and Gealy, 24-26 Mossbury Rd. London. 59 pp.

Skelton, F. and Walker, L. (1995) Pilot Study to Assess the Benefits of Gathering Evidence of Research Competencies for PhD Students to Improve Their Subsequent Employability. Glasgow University. 21pp.

Willsdon, C.A.P. (1990) ‘Dyce in camera’: new evidence of his working methods. The Burlington Magazine. Vol. 132, No. 1052, Nov. 1990, p 759-765..

Willsdon, C.A.P. (1996) Klimt’s Beethoven Frieze: Goethe, Temelkunst and the fulfilment of wishes. In Image: Music: Text. Eds. M. Pointon and P. Binski. Art History book issue. Vol. 19, March, p 44-73.

Willsdon, C.A.P. (1997) The Mural Decoration at the Royal Exchange in The Royal Exchange. Ed. A. Sunders. London Topographical Society. p 311-335.

Glasgow University Library Publications

Ashworth, S.(1998) Starting Research in Engineering. Glasgow University Library Guide. http://www.lib.gla.ac.uk/Docs/Research/engineering.html

Faichney,A.(1998) Starting Research in Life Sciences. Glasgow University Library Guide. http://www.lib.gla.ac.uk/Docs/Research/lifesci.html

Faichney, A. (1998) How to use JCR (Journal Citation Reports). Glasgow University Library Guide. http://www.lib.gla.ac.uk/Resources/Databases/jcrs.html

Marlborough, H.S. (1998) Introduction to Database Searching. Glasgow University Library Guide. http://www.lib.gla.ac.uk/Docs/Guides/searching.html

Moore, J.N. (1998) Starting Research in Psychology. Glasgow University Library Guide. http://www.lib.gla.ac.uk/Docs/Research/psychology.html

Worlledge-Andrew,H.(1998)Starting Research in UK Law. Glasgow University Library Guide. http://www.lib.gla.ac.uk/Docs/Research/uklaw.html

Appendix A - Some Abstracting And Indexing Journals

Introduction As a start, connect to BIDS at http://www.bids.ac.uk, which contains a selection of databases, including the multidisciplinary group of ISI Citation Indexes. Click on the name of the subject database you wish to search. Other databases are available via sources such as EDINA or OCLC . A password supplied by library staff will usually be required. Once the password is issued you may be able to access information from outside the library or off-campus. Note that the popular BIDS-ISI service will be superseded from 2000 by the same data as ISI Web of Science.

The following selected list is by no means exhaustive:

Arts/History/Languages/International  ABELL:Annual Bibliography of English Language and Literature. Print from 1920.  ABREES: Abstracts Russian and East European economic affairs.  Arts and Humanities Citation Index from BIDS-ISI Citation Indexes,1981  British Humanities Index- print or database.  Hispanic American Periodicals Index, in print and database (from EUREKA)  Historical abstracts-print from 1955, and database.  MLA International Bibliography- print or database, covers literature, languages, linguistics  Periodicals Contents Index- database as PCI from EDINA. Indexes range of arts journals from nineteenth century onwards.

Economics/Politics/Sociology/Anthropology  Applied Social Sciences Index and Abstracts, printed version, (database available as ASSIA Plus).  International Bibliography of the Social Sciences-covers economics, sociology, politics and anthropology.-in print- or from BIDS as IBSS.  International Development Abstracts, in print , and online in Geobase CD- ROM.  International Political Science Abstracts, available since 1951 in printed version.  Journal of Economic Literature, available in paper copy since 1969 and on CD-ROM as Econlit.  Social Sciences Citation Index from BIDS-ISI  Sociological Abstracts (Good international coverage), in print or database.  Sage Urban Studies Abstracts, available since 1987, printed version only.

Education  ERIC- Educational database, U.S. oriented.  British Education Index, available since 1954, printed version is solely an index, citations give no summary of content. As database, also available within International ERIC. Engineering/Mathematics  Applied Mechanics Reviews, available since 1948 in printed version. (Review articles with extensive bibliographies of the literature).  Aqualine Abstracts. Print or database (World’s scientific literature on water {but excludes aqua vitae and uisga beatha})s  Engineering Index in print = Compendex database available through BIDS/EDINA  Index to Scientific and Technical Proceedings available via BIDS-ISI Citation Indexes  INSPEC. Database via BIDS/EDINA or CD-ROM. Combination of print indexes Computer and Control Abstracts/Electrical and Electronics Engineering Abstracts.  Mathematical reviews, available on paper, or online as MathSci(Net)  MathDatabase/ Zentralblatt fur Mathematik/ Mathematics Abstracts. 1931onwards. See http://www.emis.de/cgi-bin/MATH/

Law  Current Law:UK up-to-date printed digest of cases and legislation  Current Legal Information, electronic version of Current Law, Daily Law Reports, UK Legal Journals Index.  European Current Law: monthly digest (Both E.U. and national levels, digest of legislation and cases, tables of competitive decisions).  Index to Foreign Legal Periodicals database available via EUREKA.  Index to Legal Periodicals available via OCLC. (Mainly U.S. and Canada).  Lawtel -UK collection of legal databases including cases and legislation.

Management/Business  Business Periodicals On Disk (BPO), on CD-ROM since 1987.  ABI/Inform database. ABI is database of first choices for management/financial administration.  Institute of Management International Databases Plus (IMID) available on CD-ROM  International Directory of Company Histories -print-available since 1988.  OneSource UK Private+. 10 years data on 110,000 companies - CD-ROM. database.  Wilson Business Abstracts (Mainly U.S. business magazines and trade publications).

Music (Helpful titles in tracing music)  The New Grove Dictionary of Music and musicians.  WorldCat, a catalogue of records submitted by libraries worldwide from OCLC databases.  Popular Titles and Subtitles of Musical Compositions  Music Index, print and database

Theology/Philosophy  ATLA Religion Database- in print as Religion Index  Philosophers Index-print and database. Science/Health (Human and otherwise)  Astronomy and Astrophysics Abstracts, available since 1969, print version.  Biological Abstracts , printed, since 1926. On-line via EDINA or CD-ROM as BIOSIS.  CAB Abstracts. In print - 29 individual abstract journals in agriculture, animal  Chemical Abstracts available since 1907. Print and database. Covers biochemistry, microbiology, pharmacology, chemistry.  Geobase is recommended database for Geography, Georef for Geosciences .  Index Medicus, varying print titles from 1879. On-line as MEDLINE since 1966 , covers medical dental, nursing and veterinary material - U.S. bias.  Physics Abstracts, in paper form only from 1964, thereafter available as part of INSPEC database.  Science Citation Index available on BIDS-ISI  Psychological Abstracts as printed version. Database as PsycLIT for coverage of books (1987-) and journal articles(1887 onwards).  Zoological Record, available since 1864 in print , or CD-ROM database. Appendix B - Search Engines And Web Sites

Kenny Forte, Paisley University. (June 1998)

British Academic Related Websites National Information Services and Systems http://www.niss.ac.uk/ Comment: Good for keeping up to date with current HE events.  HE institutions & conferences  Museums  Bookshops  Newspapers  magazines

BUBL Information Service http://bubl.ac.uk/ Comment: Good starting point when beginning a new search for information.  journal abstracts  catalogue of Internet resources arranged in subject order  banks, government departments, charities, hospitals, churches  UK registered companies  UK web directories (G.O.D. UK, Hensa, Lifestyle UK, UK Directory, UK Index Ltd, UK Plus, Yahoo UK, Yell (Electronic Yellow Pages)).

General WWW Search Engines Information about the major search engines can be found here: www.searchenginewatch.internet.com/facts/major.html  Currently the most popular search engine, as rated by professional researchers, is AltaVista (used for 45% of searches). HotBot came in at 20%, and Excite and Infoseek both rated 14%. The source for this information is: http://searchenginewatch.internet.com/resources/reviews.html  Search engine tutorials (articles and resources to help you better use search engines to find what you are looking for): http://searchenginewatch.internet.com/resources/tutorials.html Dogpile http://www.dogpile.com/ Comment: Very good site since it searches other search engines for you. Can be very slow, works best in the morning, as users in the U.S.A. are not yet on-line. Internet resources searched by Dogpile:  Usenet: Reference, Dejanews, AltaVista and Dejanews’ old Database.  FTP: Filez and FTP Search. (Only the first word will be passed on to FTP Search.)  Stock Quotes: Enter Tickers or Company Name  Business News: Search for Business News. Africa News, Agence France , M2 Airlines, Asiainfo, Business Wire , Canadian Corp , Content Factory, Fednet, Infolatina, Inter Press, Interactive Sports, Itar-Tass, M2, Phillips, PR News, PIO, Resource News, SABI, UPI, UPI, US Newswire, Washington Tech, WENN, Xinhua  Other News Wires: Yahoo News Headlines, Excite News and Infoseek NewsWires. Excite Alta Vista HotBot InfoSeek Yahoo Eureka FTP Search Global Online Directory Lycos

Searching for Softwareg2057 www.tucows.cableinet.net Comment: Good fast site for freeware and shareware software

Subject Gateways

These are UK-produced collections of quality evaluated websites aimed at the academic community.

ADAM- Art, Design, Architecture & Media Information Gateway AHDS- Arts and Humanities Data Service Biz/ed -Business and economics on the Internet EEVL-Edinburgh Engineering Virtual Library OMNI- Organising Medical Networked Information SOSIG- Social Science Information Gateway

Evaluation of Information Websites - A Selection

Resources on Evaluating Information on the Internet Evaluation of information sources(VirtualLibrary) Evaluating Library Resources