A Policy and Technical Consultation on Regulating Processes for Endorsement of Textbooks and other Support Material Run by Awarding Organisations
Information pages
About you*
Your details:
Name:
Position:
Name of organisation or AQA group (if applicable):
Address:
Email:
Telephone number:
Would you like us to treat your response as confidential?* If you answer yes, we will not include your details in any list of people or organisations that responded to the consultation.
( ) Yes (X) No
Are the views you express on this consultation an official response from the organisation you represent or your personal views?*
( ) Personal views
(X) Official response from an organisation/group (please complete the type of responding organisation tick list)
If you ticked ‘personal views’, which of the following are you?
( ) Student
( ) Parent/carer
( ) Teacher (but not responding on behalf of a school or college)
Ofqual 2014 1 A Policy and Technical Consultation on Regulating Processes for Endorsement of Textbooks and other Support Material Run by Awarding Organisations
( ) Other (including general public) (please state capacity)
If you ticked ‘official response from an organisation/group’, please respond accordingly:
Type of responding organisation*
(X) Awarding organisation
( ) Local authority
( ) School/college (please complete the next question) (
) Academy chain
( ) Private training provider
( ) University or other higher education institution
( ) Employer
( ) Publisher of resources
( ) Other representative group/interest group (please skip to type of representative group/interest group)
School/college type
( ) Comprehensive/non-selective academy
( ) State selective/selective academy
( ) Independent
( ) Special school
( ) Further education college
( ) Sixth form college
( ) None of the above (please state what)
Ofqual 2014 2 A Policy and Technical Consultation on Regulating Processes for Endorsement of Textbooks and other Support Material Run by Awarding Organisations
Type of representative group/interest group
( ) Group of awarding organisations
( ) Publisher
( ) Employer/business representative group
( ) Subject association/learned society
( ) Equality organisation/group
( ) School/college or teacher representative group
None of the above (please describe the nature of your group)
Nation*
(X) England
(X) Wales
( ) Scotland
(X) Northern Ireland
( ) Other EU country (please state which)
( ) Non-EU country (please state which)
How did you find out about this consultation?
() Our newsletter or another of our communications
( ) Via internet search
( ) From our website
( ) From another organisation (please state below)
(X) Other (please state below)
Ongoing dialogue between AQA and Ofqual
Ofqual 2014 3 A Policy and Technical Consultation on Regulating Processes for Endorsement of Textbooks and other Support Material Run by Awarding Organisations
May we contact you for more information?
(X) Yes
( ) No
*Denotes mandatory fields
Ofqual 2014 4
Questions
A. To what extent do you agree or disagree that the endorsement of resources by awarding organisations for the teaching and learning of qualifications should be allowed.
(X) Strongly agree
() Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA conducted market research in 2012 and 20131 to establish teachers’ views on the need for resources endorsed by awarding organisations. It is clear from the research that teachers value endorsed textbooks as part of the wide range of resources they can draw on. They want to be confident that a textbook is relevant to the specification they have chosen and value endorsement because this helps them select the best resource for their students and to match their teaching preferences. The professional judgment of the teacher plays an essential part in determining the relevance and appropriateness of a particular textbook for a given group of students.
Resources that are endorsed (or ‘approved’) by awarding organisations provide a number of advantages. According to AQA’s research these include:
• Initial support with a new specification so that teachers understand its scope and range • Support for new teachers and non-specialists • Support during cover lessons • Student motivation, resulting from an assurance that an endorsed textbook will not omit any knowledge essential to pass the qualification • Provision of a complete “core”, giving confidence to explore beyond this with relevant articles and further information • Easy to manage – i.e. a set of curriculum-matched resources ‘all in one place’ • General teacher and student reference
In the 2012 research, 66% of GCSE and 71% of A-level teachers requested one or more resources endorsed by the awarding body for their subject. In 2013, 66% at GCSE continued to want endorsed resources. The number at A-level had declined
1 2012 research – 71 in-depth interviews with heads of department and an email survey which received 682 responses; 2013 research – 41 in-depth interviews with heads of department, 8 focus groups and an email survey which receive 774 responses.
5
slightly to 62% though this may partly be due to the inclusion of different subjects, such as Art & Design, for which teachers traditionally draw on a much more diverse range of stimulus materials. For some subjects though there is particularly strong need for endorsed resources. In the case of Science for example, 73% of GCSE and 70% of A- level teachers requested them in 2013. Based on this research, AQA has therefore concluded that there is a clear requirement for resources endorsed by the relevant exam board in the market.
B. To what extent do you agree or disagree that we have correctly identified the risks that endorsement creates.
( ) Strongly agree
(X) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA agrees with the potential risks that endorsement could create identified in Ofqual’s Textbook Action Plan.
C. To what extent do you agree or disagree that where an endorsement process is set up, the controls we are proposing are appropriate to manage these risks sufficiently.
( ) Strongly agree
(X) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA believes that the risks posed to standards and to a healthy qualifications market are adequately managed by the new guidance in relation to Conditions A4 and F2.
The risk posed to public confidence would be suitably addressed by the proposed controls, providing the public fully understands the risks to good teaching and learning
6
that would be posed by a lack of endorsement by awarding organisations. We are concerned however that this may not be adequately understood (see our response to Question H, below, for more details).
The risk regarding the effectiveness of learning focuses on whether these materials are too geared to helping students prepare for exams rather than supporting learning. The proposed controls would address this risk for most resources written by senior examiners (endorsed and non-endorsed).
There is however still a risk that other resources by former senior examiners, non- senior examiners and experienced authors may still be very exam-focused. No controls are currently proposed to address these resources (although we acknowledge that it would be difficult to design and implement such controls).
Assessment organisations do not – and could not – have the resources needed to fully investigate all non-endorsed resources written by senior examiners. AQA contractually requires examiners themselves to disclose all authoring details. In theory it would be possible for a senior examiner to breach his or her contract and publish an exam- focused revision guide without our knowledge; however, based on our experience of working with senior examiners, we believe that the risk of such incidents occurring is extremely low. AQA would take action in the event of any such contractual breach.
D To what extent do you agree or disagree that the draft new guidance in relation to Condition A4, about conflicts of interest when a senior examiner also prepares resources for a qualification, is appropriate?
( ) Strongly agree
( ) Agree
(X) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA supports the proposed approach that senior examiners are contractually required not to engage in any personal, business or professional activity which conflicts or could conflict with their role as a senior examiner. We also agree that they should be required to inform the awarding organisation when they will be involved in the preparation of resources to support the preparation of learners. This is in line with AQA’s current practice.
We believe that senior examiners should be allowed to author resources, whether commissioned by the awarding body or a third party, with appropriate safeguards in place. However we believe that it is not feasible to require awarding bodies to monitor all (non-endorsed/approved) resources produced by all senior examiners, as implied in the draft guidance. The resource implications would be very substantial, disproportionate to the potential risk, and detract from the resources currently deployed
7
to provide other materials for teachers and students. As detailed in our response to question C, we rely on our senior examiners to inform us (as contractually obliged) about resources for which they are the author. Our assessment development process of course includes robust safeguards to prevent inappropriate levels of predictability between series.
We agree that awarding bodies should be expected to take all reasonable steps to monitor the known support materials, particularly approved or endorsed materials, produced by senior examiners against the assessments they have produced, but argue that the wording of the guidance should be modified to clarify that there is not an expectation that exam boards review every published resource on the market.
E To what extent do you agree or disagree that the draft new guidance in relation to Condition G4, about maintaining confidentiality of assessment material, is appropriate?
( ) Strongly agree
( ) Agree
( ) Disagree
(X) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA agrees that confidentiality of assessment materials is paramount and we have robust procedures in place to assure this. We are however concerned that the draft guidance would impose requirements that would not be practically achievable, and believe that three issues need to be addressed.
The first issue is the inclusion of former staff and contractors in the descriptions, and the content of the current condition G4 which relates to the ability of the boards to legally restrict the trade of an individual. Question papers are prepared up to 18 months prior to the exam and it is not possible to enforce any sort of contractual clause restricting what former staff or contractors can do for this time period after they have left the board’s service. While we can require that they inform us of the activities they will undertake once they leave the organisation again it is not practicable for us to enforce this obligation. The only reasonable step awarding bodies can take is to include a confidentiality clause in the contract; AQA does this as a matter of course and we strongly support this requirement.
The second issue concerns the requirement for all staff and contractors notify the exam board of all instances of their involvement in the preparation of resources. If this were applied to internally-produced materials, it would result in a costly and hugely bureaucratic exercise that would not contribute to meeting the primary objective of the proposed changes. We suggest that the wording be amended to make clear that this requirement applies to materials being prepared for third parties only.
8
The third issue is, as for Condition A4, that it is not feasible for awarding bodies to be required to monitor all (non-endorsed/approved) resources produced by all senior examiners, as implied in the draft guidance, due to the resource implications and the impossibility of awarding bodies having oversight of every resource published in the market.
We support the guidance which references the training of relevant staff on how to protect confidential assessment materials. We ask for clarification as to whether it is intended that this should also be applied to relevant contractors, as we believe that it is more appropriate for exam boards to provide guidance to contractors rather than formal training, as already included in their contractual obligations.
F To what extent do you agree or disagree that the draft new guidance in relation to Condition F2, about packaging qualifications and resources together, is appropriate?
( ) Strongly agree
(X) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA agrees that it is appropriate for awarding bodies to have and follow a policy that prohibits inappropriate packaging of qualifications together with other products or services.
We do not package any charged-for published resources with our qualifications. Approved resources are referred to on the AQA or AQA Family (Teachit, Alfiesoft and Exampro) websites or in marketing communications in order to inform teachers that these resources are available, as are others, to support learning – but not to promote them as required or essential purchases for the specifications, which are the definitive documents setting out the knowledge and skills required to take the assessments.
We believe that this is the correct approach and support the introduction of guidance that would oblige all exam boards to act in a similar way.
We ask for further clarification of the definition of a “negative impact” on students or purchasers. We would support this stipulation if it were defined in terms of competition law, i.e. uncompetitive or market-distorting behaviour, and would suggest that the draft language be amended to make this clear.
G To what extent do you agree or disagree that the draft new condition C3 and
9
related, about awarding organisations’ arrangements with publishers, is appropriate?
(X) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
( ) Don’t know/no opinion
Please provide comments or evidence to support your answer
AQA is supportive of the proposed new condition. We do not currently publish our criteria for approval/endorsement but we are supportive of this proposed requirement and would be happy to do this. In all other respects we believe that our current procedures represent best practice, and agree that the proposed new condition would help to ensure that such practice becomes commonplace.
The AQA Approval Form (see Appendix C) encompasses our current criteria for endorsement. We support the proposed requirement for this to be publicly available and explicitly consider any risks posed to qualification standards. We will review the form in light of this and publish the revised version on our website, and anticipate that this will be in place for Phase 2 subjects in the current round of reforms. We would urge Ofqual to allow sufficient time when implementing the new condition for this work to be undertaken properly.
We have a clear process for engagement with publishers seeking endorsement, which is handled exclusively by AQA’s Publishing Manager. We support the intention to provide clear guidelines to staff and contractors on this matter.
We are supportive of the requirement to take all reasonable steps to ensure that endorsement is signalled in the same way for all endorsed resources, as this will provide clarity to teachers and students and help to mitigate some of the risks identified by Ofqual. With the exception, for a specific reason, of some resources for the new GCSE English specification in 2010, all publishers of AQA-approved resources have since 2007 been required to display our approved roundel in the same way.2
We review the marketing materials for AQA-approved resources produced by publishers for conflict of interest phrases and content. We are confident that any marketing materials produced for AQA approved resources since 2012 do not carry any implication that the resources contain privileged information available only to our examiners or are needed in order to successfully complete an assessment or qualification. We will be ensuring that this wording is included in the next version of our conflict of interest guidance for publishers, and would support the requirement for all
2 From 2007-2011 our ‘AQA approved’ roundel was displayed in the same way on all endorsed resources published by Nelson Thornes for AQA. Since the contract with Nelson Thornes expired in 2011, publishers entering the AQA approval process have been required to display the roundel in the same way. We did however approve resources by other publishers for the new GCSE English specification in 2010, but for copyright reasons with Nelson Thornes they were unable to use the identical roundel and therefore had to create their own.
10
exam boards to take a similar approach.
Exam boards do not have the disproportionate resources that would be needed to check that our selected publishers supply us with all marketing materials that they produce. Web material, for example, can change overnight and without vast resources waiting to be deployed for this it would be impossible for AQA to keep abreast of all marketing material for approved resources on an ongoing basis. In addition, not all marketing is published and therefore susceptible to checking: a representative from an approved publisher could market their resources in an inappropriate manner during the course of a school visit. Apart from providing the publisher with initial training, AQA has very limited control over such situations. As such, we would seek confirmation that “all reasonable steps” would include all materials of which the exam board was notified by the publisher (as contractually required), but would not include a requirement for exam boards to monitor all the activities of publishers.
H. To what extent do you agree or disagree that public confidence in these arrangements will be improved as a result of the proposals.
( ) Strongly agree
( ) Agree
( ) Disagree
( ) Strongly disagree
(X) Don’t know
Please provide comments or evidence to support your answer
We believe that public opinion is likely to be divided as a result of these proposals. Exam boards and educationalists, for example, are likely to understand the underlying rationale. Publishers seeking awarding body endorsement are likely to be concerned that the proposed arrangements appear to favour those outside the approval process who can continue to produce very exam-focused books to gain market share.
Teachers may initially be pleased that senior examiners can continue as authors and that textbooks can be approved by awarding bodies. Upon publication, they are likely to realise the full impact of these arrangements. Many textbooks will have less exam practice – particularly those approved by exam boards and those written by senior examiners. This is most likely to cause issues for the publishers of these books as teachers opt for other resources, as well as for senior examiners (see reasons outlined in our response to Question L).
The general public may be surprised that senior examiners can continue to be the author of textbooks. It is therefore important that Ofqual clearly communicates the rationale for this to the media and other stakeholders to avoid confusion arising from misunderstanding of the context.
I. Are there any other alternatives to introducing regulatory controls that we
11
should be considering for endorsement processes?
The use of awarding body names in the title of books is a major issue where the awarding body has had no involvement. Many teachers understand the difference between ‘endorsed/approved’ resources and those written ‘for’ an awarding body specification. There are however teachers and students who mistakenly believe that all books written ‘for’ an awarding body specification have been endorsed or approved by the awarding body in question. This leads them to buy non-approved books under false pretences. When asked about their decision, they often say that they though the book was endorsed as the exam board’s name had been used.
Prohibiting the use of awarding body names on non-endorsed books would be both difficult and prohibitively expensive to administer. AQA proposes an alternative approach: for prominent communications via awarding bodies’ websites and other channels, as well as on the resources themselves, to explain endorsement and list all endorsed resources.
J. What criteria for endorsement would you like exam boards to use to improve the quality of endorsed resources?
AQA operates a very thorough approval process:
• Publishing proposals submitted by publishers for approval are reviewed against detailed publisher selection criteria. The proposals are analysed and evaluated at an internal meeting involving AQA’s subject experts. In most cases, publishers are then asked to submit additional information to aid our decision-making process. We know from research that it is confusing for teachers if awarding bodies approve all the resources that are written for a specification. For this reason, only two or three publishers are then selected to enter the AQA approval process for each specification. • We then meet with the selected publishers individually to discuss and challenge their publishing proposal. This enables us to understand the publisher’s proposal in detail and determine whether it is fit to enter the approval process. We do not run general meetings which all selected publishers attend. • We currently approve textbooks, followed by the digital version of the student book at a later date. We do not approve teacher guides, study aids or revision guides. • Our approval process consists of four stages: concept, manuscript and two proof stages for each resource. The process is based on our approval form, which consists of detailed questions about quality and suitability, for each stage of the process. • As material is supplied by the publisher, each stage of the form is carefully completed by our reviewer(s), signed by AQA, and returned to the publisher and author(s). AQA makes separate checks of the content for conflict of
12
interest phrases and returns these findings to the publisher. • At each stage, the publisher and author take our reviewer’s comments on board, and only when the process is completed to the reviewer’s satisfaction is the approval form signed by AQA and the resource is ‘AQA approved’.
This provides us with a robust approval process. However, and in the light of this consultation, we will continue to keep it under review and incorporate improvements where indicated in our response. We believe it would be beneficial for all exam boards to follow a similar process.
The criteria used to judge endorsement can be seen in Appendix C: The AQA Approval Form.
Equality impact assessment
We have not identified any aspects of the proposed changes to our Conditions or guidance that may have a negative impact on students because of age, gender reassignment, pregnancy and maternity, religion or belief, sex, sexuality, racial group, marital status, dependents or disability.
Question K
Are there any specific positive or negative impacts on people who share particular characteristics22 that we should consider in relation to these draft Conditions? If so, what are they and how could we address any negative impacts?
We are not aware that any of these draft Conditions would impact on any of the above groups.
Regulatory impact assessment
Question L
Would any of our proposals have financial or wider resource consequences, positive or negative, for schools, exam boards, publishers or others? Please provide evidence to support your answer.
AQA believes that there are wider consequences for publishers, teachers, senior examiners and awarding bodies.
There is a risk that publishers who provide textbooks with a notable exam focus will gain market share at the expense of those who provide textbooks to support good teaching and learning. In particular, since material that focuses heavily on exams will
13
not be permitted in approved textbooks, publishers may conclude that it is in their interests not to seek awarding body approval and to return to exam-focused textbooks for commercial reasons.
Over time we expect that publishers could begin to feel stifled by the restrictions imposed on senior examiners as authors. Publishers may conclude that senior examiners are not commercially viable authors as they are prohibited from including the material which teachers and students naturally seek out in order to improve grades.
If this situation arises, senior examiners may opt for authoring (which is more lucrative) over assessment, thereby leading to a possible decline in the pool of talented senior examiners available to awarding bodies. To that extent the proposed controls could, if not very finely calibrated, jeopardise the quality of assessments themselves in the longer term.
The consultation also raises concerns about awarding organisations working exclusively with one publisher as this would mean that students do not have access to potentially higher quality resources. In the case of a specification with a small cohort, it can be difficult for an awarding organisation to find more than one publisher who is prepared to produce resources to support it. In some cases it is in not possible to find a publisher who is prepared to take on this commercial risk. We agree that exam boards should take all reasonable steps to secure more than one publisher for each specification, but it may on occasion be necessary to work with just one publisher if any resources are to be provided. Given Ofqual’s support for the importance of providing approved resources, we would welcome clarification that this latter approach would be acceptable to the regulator, providing it can be demonstrated that reasonable effort was made to secure more than one publisher for a given specification.
14
DECLARATION OF INTERESTS OUTSIDE AQA (Please see the back of this form for guidance on how to complete)
Part A Title: First Name: Surname: Any other names by which you may have been known: Telephone AQA PIN No: Number[s]: Email address:
Part B
Do you:
a) plan to engage in any teacher training or other support related to an AQA qualification? YES NO AND
b) have access and/or input into confidential question paper material for an AQA qualification? YES NO
If you have ticked ‘YES’ to both a) and b) above please give full details of the activity(s) you plan to engage in/are engaged in below (please continue on a separate sheet if necessary):
ACTIVITY DETAIL OF ACTIVITY DATES SUPPORTING INFORMATION
(please continue on a separate sheet if necessary)
I confirm that I have read and understood the question stated above, and I confirm to the best of my ability that the information given on this form is true and accurate.
………………………………………………………………………………………….. Signature
Print Name: Date:
Important: If your position changes after the completion of this declaration, please complete a new declaration. Further declaration forms are available from your Qualifications Developer.
PLEASE RETURN THIS COMPLETED FORM TOGETHER WITH YOUR SIGNED CONTRACT FOR SERVICES GUIDANCE ON HOW TO COMPLETE THIS FORM
Date received: Considered by: Policy compliant: Yes/No Received by: Referred to Wider Interests Group: Y/N Outcome Notified: Yes/No
Please ensure you complete BOTH parts A and B of this form.
PART A OTHER NAMES BY WHICH YOU MAY BE KNOWN: Please state any other names by which you may be known or listed in our system (e.g. maiden name, abbreviated first name etc).
TELEPHONE NUMBER: Please state the phone number(s) on which you are most easily contacted during office hours.
AQA PIN NUMBER: Please state your unique 6 digit examiner PIN number
EMAIL ADDRESS: Please state the email address which you regularly access/use
PART B TICK BOXES:
If your answer to BOTH part a) AND b) of the questions stated is YES then please continue to complete the table and signature sections below.
If your answer to EITHER part a) or part b) of the question is NO then please continue to complete only the signature section below the table.
COMPLETION OF THE TABLE:
Activity: e.g. teacher training or other support Please list all activities and the period of that activity, current and future, outside those that you have agreed to provide to AQA, which you plan to undertake.
Detail: Alongside each activity you have listed please provide a summary of the detail of that activity e.g. who the work is for, what the work covers, who the work is targeted at. Information such as session plan outlines/précis of text to be authored etc are useful examples of information to include in this area.
Dates: Please specify the dates on which this activity is planned/will commence and be fully completed e.g. 1 August 2013 – 31 July 2014
Supporting Information: Please use this section to provide supporting information to help us understand the nature of the activity stated and any relationship with AQA qualifications.
SIGNATURE: Please ensure you sign, print your name clearly in block capital letters and date the form before returning.
RETURNING THE FORM
The completed form should be returned along with your signed contract for services. You will be asked to complete a further declaration each time a contract is issued.
Once your completed form has been received by AQA we will notify you within 10 working days if any of the activity(s) you have declared do not comply with the restrictions required by Ofqual.
Choose option 2 of 2
APPENDIX B
Extract from AQA Wider Interest Policy – March 2014
Any authored text or resources must:
• Not contain content or phrases which represent a conflict of interest. This includes features such as, but not limited to: o Examiner’s Tips boxes or similar devices o phrases such as ‘how to get an A*’ o phrases on how to improve your grade o reference to a particular mark achieving a particular grade o statements of what an examiner is looking for o phrases that indicate the resource is ’all that you need’ for the course o revision tips for a particular specification o any content which could be deemed to give one student a distinct advantage over another as a result of buying the resource.
• Not contain exemplar answers, with commentaries, to exam questions • Not be materials for teacher resources, exam study aids nor revision guides • Not include material used in the creation of confidential assessment materials including source material with different questions • Not reference their role as ‘examiner’ (or similar) within the resource itself, nor in any sales and marketing communications • Not include mark schemes used in live exams.
Materials for publishers must focus on good teaching and consolidating learning and avoid explicitly focusing on the exam. Questions within a text book must not exclusively be in the form they may be asked in the exam. Mark schemes may be included to allow students to mark their own work but these should not include those used in live exams.
The above conditions apply to all resource material produced for a particular specification both by 3rd parties and AQA. This may be in either print or digital format i.e. all types of planning, teaching, learning and assessment-practice materials, such as lesson plans, teacher books, student books, interactive whiteboard resources, photocopiable resources, apps, practice tests etc.
For current resources, a senior examiner who has already authored books and is in contract with a publisher at the time they are contracted to provide services as a senior examiner, is required to inform AQA in writing of the contractual commitments they have in place with the publisher. Senior examiners are not permitted to undertake the role of a reviewer for any book supporting the same specification as one for which they have authored material.
Stag Hill House Guildford GU2 7XJ T: 0161 953 1180 Private and confidential F: 01483 300 152 to be opened by the addressee only aqa.org.uk
Dear
Approval Form for AQA 2015 specifications – Confidential
Many thanks for agreeing to act as reviewer for the published resources for this specification.
The AQA Approval process is an important part of our overall support wrapper for the 2015 specifications. It enables us to guide teachers to published resources that are a good match for our specifications and where we are satisfied with the overall quality. Further details are on page 3.
Please find enclosed: • the Key information form, including details of your contact at AQA, the Resource Co-ordinator • a summary of the AQA Approval process • stages 1, 2, 3 and 4 of the Approval form • an overview of the AQA Approval process for reviewers • a list of personnel involved in the AQA Approval process for the new specifications • guidance on conflict of interest phrases and content (to follow).
Please: • read the enclosed information and let your Resource Co-ordinator know if you have any queries at this stage • complete all sections of the appropriate stage of the Approval form when you are reviewing material supplied by the publisher. Contact details for the Resource Co-ordinator are on the Key information form • read all material supplied by the publisher carefully and complete all sections • state how improvements could be made to the resource where appropriate • mark up the plans, manuscripts or proofs with any additional comments as necessary • only sign off Stage 4 of the Approval form once you are completely satisfied.
If you require further guidance on any of the questions, please contact me directly on the number below.
AQA Education (AQA) is a registered charity (number 1073334) and a company limited by guarantee registered in England and Wales (number 3644723). Our registered address is AQA, Devas Street, Manchester M15 6EX.
Please return all the approval forms electronically at each stage to the Resource Co-ordinator in keeping with the agreed schedule and as detailed in the enclosed overview.
You will be advised of the schedule shortly by the Resource Co-ordinator.
Yours sincerely
Pippa Sweeney AQA Publishing Manager 01483 556339 Email: [email protected]
AQA Approval form for 2015 specifications v3 November 2013 2
AQA Approval form - Confidential
Key information form To be completed by the Resource Co-ordinator at the start of the process
Specification
Level, eg GCSE, A-level
Date of first teaching of specification
Date of first assessment of specification
Resource Co-ordinator
Resource Co-ordinator email
Resource Co-ordinator phone
Resource Co-ordinator postal address
Publishing Manager Pippa Sweeney
Publishing Manager email [email protected]
Publishing Manager phone 01483 556339
AQA Approval form for 2015 specifications v3 November 2013 3
Publishing Manager postal address AQA, Stag Hill House, Guildford, GU2 7XJ
AQA Approval form for 2015 specifications v3 November 2013 4
Summary of the AQA approval process for the 2015 specifications
The AQA Approval process is an important part of our overall support wrapper for the 2015 specifications. It enables us to guide teachers to published resources that are a good match for our specifications and where we are satisfied with the overall quality.
The publisher is ultimately responsible for the overall quality and editorial control of the resource, but please provide guidance on the accuracy of content wherever possible. In addition, the publisher will ensure that the resource is proofread, but please mark up any typographical errors you find. AQA’s Publishing Manager will also check for ‘conflict of interest’ phrases – eg ‘How to get an A*’; ‘you will be asked xyz in the exam’ etc. Please also mark these up if you see them in the 1st and 2nd proofs. See Appendix 3 for details.
The following grid outlines the AQA approval process for the 2015 specifications. We are keen to learn lessons for resourcing future specifications, so please jot down any ideas you may have in the course of the process. We will ask you for your input once the process is completed.
Approval Approval Activity Managed by process stages format
Initial stage 1 F2F visit with Initial notification of AQA Publishing Manager publisher new specification
Initial stage 2 Email Expression of interest Publisher
Initial stage 3 Emails Non-Disclosure Publisher Agreement (NDA) agreed and signed
Initial stage 4 Emails Publisher information AQA Publishing form supplied by AQA Manager; Qualifications providing basic Developer (QD) to information about the complete form new specification
Initial stage 5 Various Reviewer recruited and AQA Publishing Manager contracts signed with colleagues in the General Qualifications Division (GQD)
AQA Approval form for 2015 specifications v3 November 2013 5
Proposal stage 1 Proposal Publishing proposal Publisher developed and submitted to AQA
Proposal stage 2 Email Rejection or entrance AQA Publishing confirmation to the AQA approval Manager, QD and process Reviewer
Review stage 0 Verbal/written Initial phone or email Reviewer and Publisher as appropriate discussion if required
by Publisher – though often not
Review stage 1 Approval form Review of content Reviewer plans Resource Co-ordinator, (Content then Publisher* developed by publisher)
Review stage 2 Approval form Review of manuscript Reviewer Resource Co-ordinator, AQA Publishing Manager (for conflict of interest work) and Publisher*
Review stage 3 Approval form Review of 1st set of Reviewer page proofs Resource Co-ordinator, AQA Publishing Manager (for conflict of interest work) and Publisher*
Review stage 4 Approval form Review of 2nd set of Reviewer page proofs Resource Co-ordinator, AQA Publishing Manager (for conflict of interest work) and Publisher*
Review stage 5 Email Review of 3rd set of Reviewer proofs (NB only if Resource Co-ordinator, required) AQA Publishing Manager (for conflict of interest work) and Publisher*
AQA Approval form for 2015 specifications v3 November 2013 6
Payment Email and forms Reviewer submits Reviewer, AQA
expenses, paid by Publishing Manager,
AQA, publisher AQA Finance, Publisher
invoiced by AQA,
publisher pays AQA
Final approval Approval AQA receives final AQA Publishing Manager certificate Approval form to sign. If and Head of Subject or
approved, publisher senior colleague
receives certificate of
AQA approval
* The Publisher sends the content plans, manuscript and sets of electronic proofs direct to Reviewer, AQA Publishing Manager and the AQA Resource Co-ordinator. Publishers must also supply hard copies of proofs for the AQA Publishing Manager and reviewers upon request.
AQA Approval form for 2015 specifications v3 November 2013 7
Stage 1: Review of content plans
To be completed by the Reviewer
Please read the material supplied carefully and complete all sections.
Specification
Level (eg AS/A2)
Reviewer
Resource Co-ordinator
Date completed by Reviewer
Content
To what extent does the proposed scope of the resource cover the specification?
Is the amount of content proposed for each part of the specification appropriate?
How do you rate the proposed learning sequence? [Please rank out of 10, where 10 = excellent]. Is it appropriate?
Is it suitable for all ability ranges, or if it is differentiated, is it suitable for the level identified?
AQA Approval form for 2015 specifications v3 November 2013 8
Is there suitable scope for learning the subject beyond the specification? eg references to additional resources (online, print, audio, video etc) or extension exercises (NB may not apply to all subjects)
Are the learning objectives/outcomes clear and appropriate to the specification?
Resources
Are the proposed formats for the delivery of the learning, teaching and assessment practice resources appropriate?
Conclusion
Would you recommend that AQA approve this plan so that the next stage of the project can begin? If not, please list exactly what needs to be changed before this approval can be given.
Reviewer signature: ______Date: ______
Qualification Developer signature: ______Date: ______
AQA Approval form for 2015 specifications v3 November 2013 9
Stage 2: Review of manuscript
To be completed by the Resource Co-ordinator and Reviewer
Proofs to also be read for ‘conflict of interest’ phrases by the Publishing Manager and commented on
Resource Co-ordinator Please complete
Specification
Level (eg AS/A2)
Reviewer
Qualifications Developer
Date completed by Reviewer
Special notes Are there any special notes to include at this stage? eg reviewer’s holiday; publisher has advised that page x needs reworking; whether anybody else is involved in the review; the Lead Resource Developer has raised issues which impact on these resources etc
AQA Reviewer Please read the material supplied carefully and complete all sections.
Content NB Several questions are repeated from the Content Plan stage as you may have additional comments at this point. Layout and design are covered in Stage 3.
To what extent does the
AQA Approval form for 2015 specifications v3 November 2013 10
proposed scope of the resource cover the specification?
Is the amount of content proposed for each part of the specification appropriate?
Are the facts, theories and models all correct? Are they all appropriate for the level? Are they all relevant?
Is the selection of texts, images, case studies appropriate? (NB Not all images etc will be present at this stage)
Is there suitable scope for learning the subject beyond the specification? eg references to additional resources (online, print, audio, video etc) or extension exercises (NB may not apply to all subjects)
If this is a blended resource, are there any references to online material which you believe would be better in the text (or vice- versa)? Please give details.
AQA Approval form for 2015 specifications v3 November 2013 11
Is there any content which is likely to date quickly? (eg sporting events, images, notably old data etc). If so, do you have any suggestions as to how this might be overcome? (NB Not all of this content will be present at manuscript stage)
Is there any content which might be misinterpreted and cause offence?
Are there any references to AQA, examiners or exams which you think may represent a conflict of interest? (eg ‘Here’s how to get an A*’; ‘this book is written by the people who mark your exams’ etc) (NB The material will be checked for this again by AQA’s Publishing Manager, but please raise anything which concerns you here. See Appendix 3)
AQA Approval form for 2015 specifications v3 November 2013 12
How would you describe the proposed exercises? ie Are there sufficient exercises? Are there an appropriate number of questions per exercise? Are they suitable for the level and for the ability range?
Is the resource(s) consistent in terms of the level of the specification? Are there any pages which give cause for concern? Are any texts too lengthy or short? Are diagrams too demanding or too simplified, for example?
Is the level of language appropriate for the ability level?
Overall, is the content easy to use? (eg with good use of innovative graphics, flowcharts, diagrams etc to improve accessibility for teachers)
AQA Approval form for 2015 specifications v3 November 2013 13
Resources
Are the proposed resources for assessment practice appropriate? Do they promote good teaching/learning practice? NB Assessment practice should not be based on AQA’s Specimen Assessment Materials as these do not provide an additional resource for teachers. SAMs are freely available on the AQA website.
Are the proposed formats for the delivery of the learning, teaching and assessment practice resources appropriate? (eg interactive eBook, printed textbook etc)
Conclusion
Would you recommend that AQA approve this manuscript so that the next stage of the project can begin? If not, please list exactly what needs to be changed before the publisher can proceed to the next stage.
Reviewer signature: ______Date: ______
Qualification Developer signature: ______Date: ______
AQA Approval form for 2015 specifications v3 November 2013 14
Stage 3: Review of 1st proofs
To be completed by the Resource Co-ordinator and Reviewer
Proofs to also be read for ‘conflict of interest’ phrases by the Publishing Manager and commented on
Resource Co-ordinator Please complete
Special notes Are there any special notes to include at this stage? eg reviewer’s holiday; publisher has advised that page x needs reworking; whether anybody else is involved in the review; the Lead Resource Developer has raised issues which impact on these resources etc?
AQA Reviewer Please read the material supplied carefully and complete all sections.
Coverage / compliance
(ie with the final specification)
To what extent does the material comply with the final version of the AQA specification?
Changes since Stage 2
Please review the Outcome of Stage 2.
AQA Approval form for 2015 specifications v3 November 2013 15
Have the issues that were raised (at the bottom of page 10) at Stage 2 been addressed? Are there any additional discussions or notes which need to be included here?
Content IMPORTANT: Please review any content, images, diagrams etc which were not included at manuscript stage (Stage 2).
Is the selection of texts, images, case studies appropriate? Are there any texts, images etc which you would particularly advise against or recommend? (NB may not apply to all subjects) Can you see any potential copyright issues?
Is material presented consistently? (ie consistent conventions between chapters, sections etc?)
AQA Approval form for 2015 specifications v3 November 2013 16
Is there any content which is likely to date quickly? (eg sporting events, images, notably old data etc). If yes, how can this be amended?
Is the content accurate and up to date?
Layout and design
How do you rate the overall layout and design of this book? ie Is the typeface easy to read? Are the pages well balanced or too dense? Are the pages well structured or confusing?
Are the diagrams and images clear? If not, how can they be improved? Please give specific examples.
AQA Approval form for 2015 specifications v3 November 2013 17
Are innovative diagrams, flowcharts and artwork used to support accessibility for teachers and to help them deliver the content?
Key issues
Are there any key issues that must be addressed before approval can be offered?
Recommendation
Would you recommend that
AQA approve this material immediately? If not, please list the changes that must be made before approval can be granted.
Reviewer signature: ______Date: ______
Qualifications Developer signature: ______Date: ______
AQA Approval form for 2015 specifications v3 November 2013 18
Stage 4: Review of 2nd proofs
To be completed by the Resource Co-ordinator and Reviewer
Proofs to also be read for ‘conflict of interest’ phrases by the Publishing Manager and commented on
Resource Co-ordinator Please complete
Special notes eg reviewer’s holiday; publisher has advised that page x needs reworking; whether anybody else is involved in the review; the Lead Resource Developer has raised issues which impact on these resources etc
AQA Reviewer Please read the material supplied carefully and complete all sections
Please note that Q1: Have the issues that were raised and the changes that were proofreading will be required in Stages 1, 2 and 3 been addressed? carried out by the If not, please list the issues that have not been addressed. publisher during this review. This process should pick up typographical errors, so there is no need for the AQA Reviewer to focus on this level of detail.
Please note that Q2: Do the practice questions reflect the style of AQA’s specimen practice or questions for the new specification? examination-style If not, please list the changes that must be made before approval can be questions may granted. include past paper questions (some adapted / updated). Please pay special
AQA Approval form for 2015 specifications v3 November 2013 19
attention to these questions.
Recommendation Q3: Have any changes been made that would cause you to withdraw your recommendation that AQA should approve this material? … to AQA Qualifications Manager or Head of Subject
Reviewer signature: ______Date: ______
Qualifications Manager signature: ______Date: ______
AQA Approval form for 2015 specifications v3 November 2013 20
Stage 4: Review of 2nd proofs
For the Qualifications Manager
Qualifications Manager Please complete
Outcome Course of action agreed with Publisher, when applicable
Qualifications Manager signature: ______Date: ______
Please return this form to the Resource Co-ordinator.
Final sign off will be completed by the relevant AQA Head of Subject and AQA Publishing Manager on the AQA Approval Certificate.
AQA Approval form for 2015 specifications v3 November 2013 21
Appendix 1 – Overview of Approval Process for Reviewers
Stage 0 – Initial enquiries Reviewers may be asked a few initial questions by the Publisher via email or by phone prior to Stage 1. We will ask the Publisher to keep these to a minimum, but ultimately time spent on this stage will save time at subsequent stages.
Stage 1 – Content plans
At Stage 1, you will receive the electronic Approval form from the AQA Resource Co-ordinator and the electronic (or printed) Content plan from the Publisher to review.
Please: • check you have received the correct Content plan • check the Content plan carefully – this should involve both a thorough read-through, and specifically reading against the Stage 1 Approval form • complete Stage 1 of the electronic Approval form, providing specific examples to support your comments • return the entire Approval form and Content plans to the Resource Co-ordinator at AQA either electronically via Secure email (details TBC) or in the AQA envelopes which will be provided • keep copies of the completed Approval form and the Content Plan so you can refer to them at Stage 2.
Stage 2 – Manuscript
At Stage 2, you will receive the electronic (or printed) Materials to review from the Publisher.
Please: • check you have received the correct Materials • review the Materials carefully – this should involve both a thorough read-through, and specifically reading against the Stage 2 Approval form • complete Stage 2 of the Approval form, providing specific examples to support your comments • mark up any Material you receive to help clarify any comments on the Approval form • return the entire Approval form and all Materials to the Resource Co-ordinator at AQA • keep copies of the Approval form and your corrections on the Materials so you can refer to them at Stage 3.
AQA Approval form for 2015 specifications v3 November 2013 22
Stage 3 – 1st proofs
As with Stage 2:
• check that you have received the correct Materials and the Approval form • check the Materials carefully – this should involve both a thorough read-through, and reading specifically against Stage 3 of the Approval form and the Stage 2 corrections • check the Stage 3 Materials against your completed Stage 2 Approval form to ensure that all your comments have been taken on board correctly. Also check that there are no additional changes • complete the Stage 3 Approval form, providing specific examples to support your comments • mark up any Material you receive to help clarify any comments on the Approval form • return the entire Approval form and all Materials to the Resource Co-ordinator at AQA • keep copies of the Approval form and your corrections on the Materials so you can refer to them at Stage 4.
Stage 4 – 2nd proofs
At Stage 4, repeat the approval process at Stage 3, ie:
• compare Stage 4 Materials with Stage 3 • check the Stage 4 Materials against your comments on the Stage 3 Approval form and Materials • complete the Stage 4 Approval form • return the Approval form and all Materials to the Resource Co-ordinator at AQA • keep copies of the Stage 4 Approval form and Materials.
Subsequent review stages (if required)
• We do not anticipate the review process going beyond four stages. • On the rare occasions that this does happen, the Reviewer will be paid for this work. • The Reviewer will be paid by AQA, and this will be charged to the Publisher.
AQA Approval form for 2015 specifications v3 November 2013 23
Appendix 2 - Personnel involved in the AQA Approval Process for the new specifications
Personnel Roles and responsibilities
Publishing Manager AQA Publishing Manager who has overall management of the Approval Process of resources for the new specifications.
Resource A member of the AQA subject team who oversees the day-to-day approval Co-ordinator process and has direct contact with the Lead Resource Developer, the Reviewer and the Publisher.
Lead Resource A subject specialist who creates and identifies resources to support the Developer content of the new specifications.
Qualifications A member of the AQA subject team who signs off the Reviewer’s comments Developer at the first three stages of the Approval form.
Qualifications A member of the AQA subject team who signs off at the final stage of the Manager Approval form.
Head of Subject Head of AQA subject team with final sign-off along with the Publishing Manager for the Approval process for an individual resource – ie that it is ‘AQA approved’. Responsible for the delivery of the approval processes within their team to meet the quality expectations and also agreed deadlines.
Subject specialist whose task is to review, amend and approve the resource Reviewer materials created by the Publisher.
Publisher External publishing company who is responsible for the overall quality and editorial control of appropriate resource material.
Publishers: Responsible for the commissioning of new publishing for their particular Commissioning subject or area. AQA’s key contact at concept stage. Editor or Publisher
Publishers: The Production Manager (or an Editor) responsible for providing the Production Manager Reviewer, Resource Co-ordinator and Publishing Manager with the Content (or Editor) plan, manuscript and subsequent proofs.
AQA Approval form for 2015 specifications v3 November 2013 24
Appendix 3 – Guidance on conflict of interest phrases and content
Currently under discussion. Details to be supplied in early 2014.
AQA Approval form for 2015 specifications v3 November 2013 25