How to Improve Student Learning Without Spending More Time Teaching

TOC

Enhancing Engineering Education – Introductory workshop.

The Teaching Trick – how to improve student learning without spending more time teaching. Kristina Edström

The CDIO approach for engineering education development. Kristina Edström

Integrated Learning in a Project Course. Jakob Kuttenkeuler, Naval Architecture; Stefan Hallström, Lightweight Structures. Kristina Edström

Technology Enhanced Learning (TEL)/E-learning. Johan Fridell E-learning Development Manager Business Solution Owner (BSO) e-learning

Program Development and Management. Hans Havtun, Program Director Energy and Environment

Enhancing Engineering Education September 26th – 30th KTH, Stockholm Monday 26 September 09.15 – Introductions AKH,

12.00 MB 13.15 – CDIO – the Idea, Methodology and Community KE 16.00 Tuesday 27 September 09.15 – How to improve learning in student engineering projects JK, KE 12.30 13.30 – Continuation from the morning session: JK, KE 16.00 Visit to student labs, Q& A session, and group exercise. Wednesday 28 September 09.15 – The Teaching Trick – How to improve student learning without KE

12.00 spending more time teaching 13.15 – How to improve student learning in lectures – Peer FL

16.00 instruction

Thursday 29 September 09.15 – Room for learning – visiting the KTH learning environment MB 12.00 Starting point: D31

14.00 – Visiting a program. Meeting point: Brinellv.66 AKH,

16.00 HH Friday 30 September 09.15 – Course evaluation for development DB 12.00 13.15 – Results, reflections and next steps AKH

16.00 Anna-Karin Högfeldt, [email protected]

• Director of Faculty development in Teaching and Learning, KTH/ ECE • Co-author to Guide to Challenge Driven Education • Coordinator for Program Director’s Network at KTH • Currently running Engineering enhancement and development projects in Nordic countries, Brazil and Tanzania

Summary of this introduction workshop

HIGHER EDUCATION IN SWEDEN AND AT KTH HOW DO WE ENHANCE ENGINEERING EDUCATION? KTH – Sweden's leading University of technology

• Sweden's oldest and largest University of technology (1827) • More than 11,000 full-time students • More than 1,800 PhD Students • Over 4,800 employees • Students and personnel from more than 100 countries

WWW.KTH.SE Study programmes

• Technical preparatory programme (1/2 or 1 year) • 8 Bachelor of Science in Engineering programmes (3 years) • Master of Architecture (5 years) • 15 Master of Science in Engineering programmes (5 years), in general composed of a Bachelor’s programme in Swedish (3 years) and Master’s programme in English (2 years) • Programmes for a Degree in Engineering and Education

Percentage of women in different engineering programmes at KTH 1986‐2006

Programmes started 1990 or later ’

7 http://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2006/rapporter06/palm_therese_06068.pdf Swedish Government

• Granting university status • Enacting legislation regulating the higher education sector • Funding higher education courses and study programmes • Funding a high proportion of research • Appointing vice-chancellors of higher education institutions • Regulating the agencies involved in the higher education sector

All information and more can be found at www.ukambetet.se Higher Education Institutions

• There are 48 institutions offering higher education in various forms in Sweden • The majority of universities and university colleges are public authorities, subject to the same legislation and regulations as other public authorities in Sweden, as well as the particular statutes, ordinances and regulations relevant to the higher education sector. • A small number of universities and university colleges are self-governing and independent. Freedom

HEIs enjoy a great deal of freedom within the framework of the statutes, ordinances and regulations laid down by the Government. HEIs can make decisions about the following: • Organization of the HEI into units and decision-making bodies • Allocation of government funding within the organisation • Quality assurance procedures • Content and design of courses and study programmes • Number of available places on courses and study programmes • Admission and enrolment procedures • New professorships • Research focus • Contract education

Structure of Swedish higher education qualifications

An independent project equivalent to 30 credits, or two 15-credit projects

Independent project equivalent to 15 credits Qualification descriptors

All course and programme specifications must state the intended learning outcomes in terms of mastery of intellectual skills, mastery of knowledge and conceptual understanding, and must give details about teaching and learning strategies and methods of assessment. Funding of Higher Education

Detailed information can be found at: http://ukambetet.se/highereducationsystem/funding.4.4149f55713bbd917563800011054.html The following slides are directly taken from this site Numbers are based on year 2013

HEI total revenue 1st & 2nd Cycle 3rd cycle and (public purse) research 62.8 billion SEK 44% 56% (85%) A bit less than 2% of The amount of Increasingly Sweden’s GDP funding is based on financed from the number of full- indirect government time equivalent funding and external students and the sources, including annual performance the government equivalent. The research funding amount of funding body, foundations, varies depending on local government, the disciplinary county councils and domain. There is the private sector. also a funding cap.

HEI = Higher Education Institutions Funding for first and second-cycle courses and programmes and for research and third-cycle courses and programmes 2004 and 2013, SEK billions in 2013 prices. HEIs´ revenues for first and secondcycle courses and programmes and for research and third-cycle courses and programmes 2003–2013, SEK billions in 2013 prices Number of students registered in first and second-cycle courses and programmes each autumn semester 1977– 2013 Student finance

• Tuition at higher education institutions in Sweden is free-of- charge for Swedish students and for students from the European Union (EU), the European Economic Area (EEA) and Switzerland. • Everyone below the age of fifty-four has the right to apply for student finance for a maximum of 240 weeks. • Student finance is intended to cover living expenses and the cost of study material. • Student finance comprises a grant and a loan. • The student loan must be repaid on a monthly basis before the loan recipient reaches the age of sixty. • The size of the monthly payment is determined by the size of the debt and the interest rate. The amount is also adjusted to the recipient's income and ability to pay. Quality enhancement and development of Swedish Higher Education

Anna-Karin Högfeldt Director of Faculty Development at KTH ”Creating relationships and building teams, making decisions based on so much input you can get and telling them right, talking to authorities and media … well, caring about the whole situation”

A stakeholder on the expectations of an engineer – not well prepared enough for the ”swamp” of complexities in real-life

“The schools view teaching as transfer of information; learning as receiving, storing and digesting information. ‘Knowing that’ tends to take priority over ‘knowing how’; and know-how, when it does make its appearance, takes the form of science-based technique.”

Schön, Donald A. "Educating the reﬂective practitioner." San Francisco (1987). P.309. See also for instance: Benner et al 2010; Sullivan 2005; Baker 2009; Bennett et al 2000; Atkins 1999; Crawley et al 2007 Hanning et al (2012) IJSHE; 13:3, 305-20

”This study indicates that Swedish industry needs a higher level and a broader range of competences related to Sustainable development amongst all engineers than university is currently providing.

- not only environmental issues, - but additionally, sustainable business development, societal aspects and communication

Traditional education has not provided the training for graduates to work towards developing solutions to the new and complex world problems emerging.

These problems are multi-dimensional and cannot be addressed by a specific application of conventional scientific, economic or social theory.

US National Academy of Engineering, 2008 Sustaining life on Living secure Promoting Living and earth from threats healthy living learning with joy Make solar Secure Advance health Advance energy cyberspace informatics personal learning economical

Manage the Prevent nuclear Engineering Enhance virtual nitrogen cycle terror better medicines reality

Engineer the Provide energy Reverse engineer tools of scientific from fusion the brain discovery Develop carbon Provide access to sequestration clean water methods

Restore and improve the urban infrastructure Cambridge key themes. Cruickshank & Fenner, IJSHE 2012, 13:3, 249-262

• Dealing with complexity • Dealing with uncertainty • Dealing with change • Dealing with other disciplines • With environmental limiations • People • Whole life costs • Trade-offs

”Evolution of engineering education”

Problem- Traditional Challenge-based based/CDIO projects Malmqvist, J., Rådberg, K. K., & Lundqvist, U. (2015). Comparative Analysis of Challenge-Based Learning Experiences. In Proceedings of the 11th International CDIO Conference, Chengdu University of Information Technology, Chengdu, Sichuan, PR China. Recuperado de: http://rick. sellens. ca/CDIO2015/final/14/14_Pa per. pdf. • Engineering • Engineering & • Engineering • Product business Science context • Societal context • R&D context • Designing • Problem • Analyzing • Integrative formulating & • Reductionist • Team Designing • Individual • Customer • Integrative • Objective needs • Team & Individual • Value-driven Suggested definition by Malmqvist et al (2015)

“A challenge-based learning experience is a learning experience where the learning takes places through the identification, analysis and design of a solution to a sociotechnical problem. The learning experience is typically multidisciplinary, takes place in an international context and aims to find a collaboratively developed solution which is environmentally, socially and economically sustainable.” Strategies for an outcomes based and student centered learning approach to education development

Educational change

Curriculum Faculty Organization Student development development / Institutional (programs, (HE teacher development courses, module) training) development

Learning Problem Physical Democracy, outcomes crafting infrastr influence

Learning Social Values and Facilitation activities infrastr attitudes

Assessment Knowledge, Organi- SCL procedures skills, values zational infr.

Based on Mona-Lisa Dahms’ work A. Curricular level Systematic collaboration among courses/faculty in the whole educational program

How well students reach the degree outcomes has

becomeYear 1 more interesting,Intro course insteadPhysics of onlyMathematics looking I at how well one isolated course achieves its goals Numerical Mechanics I Mathematics II Methods

Year 2 Mechanics II Solid Product ‘create connections, sequences,Mechanics timing development and logical flow of assessment tasks across the whole program’Fluid Sound and Thermodynamics Mathematics III mechanics Vibrations

Signal Year 3 Control Theory Electrical Eng. Statistics analysis Cooperation among teachers, and not only on a departmental level, but across the study program’s different courses, is seen as a key step to make this happen Collaboration among teachers in program teams QUALITY OF STUDENT LEARNING

”didn’t understand”

”got it” —

passed failed exam exam [Steve Hall, MIT] What the student should learn (intended learning outcomes, ILO)

Learning Assessment Activities of Learning

Alignment in courses/programs Biggs among many Three stages in learning

1. FIRST EXPOSURE first presented with new Distributed among facts, concepts, available times: vocabulary • Class time 2. PROCESS • Students’ study students analyze, solve time problems, apply • Teacher’s own time 3. RESPONSE getting feedback from peers, teachers and Increase class time hours spent more on 2 and 3 Quotes from our students on how they spend their time

”...what did it take, five hours in ”They are quite the group and normal recitations then two hours on where he solves my own.” problems, right? I don’t prepare for that.”

Traditional recitations

Alternative recitations Systematic integration of engineering competences in the programme (p.28 handbook)

Development routes (schematic) Development levels Introductory for challenge based Year 1 Mathematics I course Physics - Complexity of task - Amount of Numerical Mechanics I Mathematics II supervision Methods and teaching - Team size Year 2 Mechanics II Solid Product Mechanics development- Deadlines - Presentation forma Fluid Sound and Thermodynamics Mathematics III mechanics Vibrations

Signal Year 3 Control Theory Electrical Eng. Statistics analysis

Oral Report Project Teamwork presentation writing management KTH's strategic partners

www.kth.se The development of Academic’s teaching skills in Higher Education is seen to be crucial in order to meet the demands B. Faculty / HE Teachers training: The Swedish context. 10 weeks / 15 ECTS The participant shall demonstrate the ability to - discuss and problematize student learning in the participant’s own subject area, on the basis of research in educational sciences and/or subject didactics of relevance for teaching in HE

- independently and jointly with others, plan, implement and evaluate teaching and assessment in higher education with a scientific, scholarly or artistic basis and within their own area of knowledge

- make use of, and assist in the development of, physical and digital learning environments to promote learning for groups and for individuals

- interact with students in an inclusive manner and demonstrate knowledge of rules and regulations regarding students with disabilities and of available student support

- apply relevant national and local rules and regulations, and to discuss society’s objectives for HE and the academic teaching role in terms of the participant’s own practice and students’ active participation in HE

- on their professional approach to academic teaching and their relationship with the students, and also towards the fundamental values of higher education, such as democracy, internationalization, gender equality, equal opportunities and sustainability

- collect, analyze and communicate their own and others’ experiences of teaching and learning practices, and relevant outcomes of research, as a basis for the development of educational practice and of the academic profession. KTH ”Teaching and Learning in Higher Education”, LH231V (Faculty Teaching Competence course, 7,5 ECTS)

Learning cycles LC1: Learning & Learning Environments

LC2: The Students

LC3: The Role of the Teacher

LC4: Designing courses to facilitate meaningful learning

LC5: Pedagogical and Professional Development Assessment tasks: Individual portfolio Team based project work (with senior academic teachers as mentors)

C. Organizational and Institutional level

All work needs to be institutionalized in the organizational structure and the way we work • A unit/department for quality enhancement and development of engineering education • Academic staff /HE engineering education teachers and researchers that are also instructional developers part of their time • Program directors’ network • Teacher Support Web: https://intra.kth.se/en/utbildning/lararstod/welcome-to-the-teacher- support-web-1.579802 • Mandatory teachers’ training • ”Open lab” environment • Strategic partners • Internal quality assurance system • External / National quality assurance system

What the student should learn (intended learning outcomes, ILO)

Assessmen Learning t of Activities Learning 2016-09-23

The Teaching Trick – how to improve student learning without spending more time teaching

Kristina Edström Deutsche Gesellschaft für Hochschuldidaktik [email protected] dghd16, Bochum, 23 September 2016

Kristina Edström Engineer & Educational developer § M. Sc. in Engineering, Chalmers § Associate Professor in Engineering Education Development at KTH Royal Institute of Technology, Stockholm, Sweden § 700 participants in the 7.5 ECTS course Teaching and Learning in Higher Education, customized for KTH faculty, 2004-2012 § Director of Educational Development at Skolkovo Institute of Science and Technology, Moscow, 2012-2013

Strategic educational development, national and international § CDIO Initiative for reform of engineering education since 2001 § SEFI Administrative Council, 2010-2013

Some publications § Crawley, E.F., Malmqvist, J., Östlund, S., Brodeur, D.R., and Edström, K. (2014) Rethinking Engineering Education: The CDIO Approach, 2nd ed., Springer Verlag § Edström, K., & Kolmos, A. (2014). PBL and CDIO: complementary models for engineering education development. European Journal of Engineering Education, 39(5), 539-555 § Edström, K. (2008) Doing course evaluation as if learning matters most, Higher Education Research & Development, 27:2, 95 – 106

1 2016-09-23

Cost-neutral interventions

To persuade the grumpy To support those dedicated professor to listen to teaching

Anyone can improve a course (at least some little bit) by working 100 hours more…

Yeah. We don’t have those hours.

And “more of the same” is probably not the most effective strategy either…

2 2016-09-23

We want to improve (maximise) student learning with a given (or reduced) level of teaching resources

Output η = Input

Then we need pedagogical know-how!

Pedagogical competence

3 2016-09-23

Or in other What should the students Formulating be able to do as a result words… of the course? intended learning outcomes

Constructive alignment [Biggs] Designing Designing activities assessment

What work is appropriate for What should the students do the students to do, to reach to demonstrate that they fulfil the learning outcomes? the learning outcomes?

Pedagogical competence

1. setting clear objectives (intended learning outcomes) o relevant for the study programs o defining the threshold level of quality o deeper working understanding 2. uphold the threshold level of quality o only pass the students who reach the goals 3. create a course which generates appropriate learning activity o so students actually reach the goals o good throughput - with good quality 4. and doing this while using teacher time effectively o generate appropriate study for the students o spend your time where it has effect on learning o create a sustainable workload for yourself o and sustainability for your institution and country

4 2016-09-23

The teaching trick

Do more of that which contributes to learning Pretty easy

But since we don’t have 100 hours more:

Do less of that which does not contribute Pretty hard

Which one is easier and which one is harder?

Examples are illustrations of principles

A specific will example illustrate

generic to principles inspire

applications - of many different kinds.

5 2016-09-23

Pick one! /* No comments */ . Family dinner . Invest 0,20 € . Seven minutes . Master test . Fireworks .

maybe later: Ultimate frisbee .

/* No comments */

6 2016-09-23

The teaching trick: Do less of that which does not contribute

Spend less time on… ”finishing” student work!

Professor S told us:

”I got 60 reports. It is a boring task to give feedback and it takes me two weeks. I gave individual comments and asked those who had failed to re-submit.

When the reports came back they were still bad. The students had only corrected the things I specifically commented on. They did not even read the rest!

Next year I did not give individual feedback on failed reports. Instead I made a list with the most common errors. Now the students had to find their own errors. When I got the reports back they were generally very good!”

7 2016-09-23

Remember the purpose

§ The purpose is not that this particular report should be good

§ The purpose is that the student should develop the skills to write reports (so that he/she can write 1000 excellent reports later)

For the same reason: Keep your hands on your back…

when you are assisting students in the computer lab – do not ever touch their keyboard!

8 2016-09-23

⏏

Every time you tie the shoes for your child, you hinder her own development. Maria Montessori

Family dinner

9 2016-09-23

The teaching trick: Do less of that which does not contribute

Spend less time on… marking coursework!

What Professor K does…

1 2 3 4 5 … Course Course start end

The weekly assignment cycle drives the course

10 2016-09-23

The weekly cycle

Feedback session i. Students papers are exchanged randomly, and they write feedback with a red pen. ii. Students receive & read their feedback immediately. iii. Advanced and lively discussions! Afterwards, teacher collects reports (or copies) for grading.

Workshop §Week Feedback 1: § Students § sessionCourse intro work on homework § Introduce § Support and new content discussions § Homework if needed

1. Read theory and implement the method (straight-forward implementation) 2. Test and verify implementation (normal use and extreme cases) 3. Investigate creatively (test variants, how would it work if…, play around, think for yourself) 4. Write short report (2 or 3 pages) (describe methodology, limitations etc and own initiatives)

Here comes the trick: Easy marking J

Grading scale • Fail = 0p (Seldom happens) • Pass = 1p (Typical grade) • Brilliant = 2p (Requires lots of own initiatives) + With accepted participation in the feed-back loop +1p

Easy to see the difference between 0, 1 or 2 points, in fact it only takes about 1-3 minutes per paper…

At the end of the course, points are converted to final grade (no exam) + In some courses there is also an oral exam

11 2016-09-23

The principle is to separate the processes – then both can be made cost-effective Feedback Assessment for for learning grading • made into a group • by the teacher learning activity • minimalistic • intense involvement • sufficiently fair • learn to discuss the subject • immediate feedback • expose variation • social motivation

Good for learning! ⏏

Continuous studies § Distributes student effort during the course.

The formative feedback session as a whole (giving feedback, getting feedback and discussions) generates learning: § Repetition – Variation – Fast feedback. § Deep & interesting discussions (instead of discussions on definitions). § Social motivation – expose your understanding to others and see theirs.

Satisfaction: § Students feel that the teacher really cares about their work. § Clear, fair and transparent grading system. § Students feel their progression.

Good for the teacher! § ≈1-3 minutes per paper. § Final grading is no extra work J

12 2016-09-23

Invest 0,20 €

The teaching trick: Do less of that which does not contribute

Spend less time on… learning activities that don’t generate appropriate study!

13 2016-09-23

⏏

The Iceberg Principle

Group work with random presenter

Day 1: All students in the group should be ready to present the whole project

Last minute: Choose the presenter randomly

Cost: 0,20 €

Seven minutes

14 2016-09-23

The teaching trick: Do less of that which does not contribute

Spend less time on… designing and correcting exams!

Oral exams are really good for learning § Better test of understanding & can be individually tailored § Affect student preparation – they know they have to show ”real” understanding, in real time (create the right expectation)

Some teachers are nervous about… ...inventing the necessary questions § The trick: Reverse the burden of proof (”the first 7 minutes are yours, to show me that you have reached the learning outcomes”)

§ Follow-up questions will pop up! …grading

§ Use a simple scale: Fail / 10p / 20p ...having to fail students § Photograph the written start for documentation

§ Ask kindly how they think it went Katrin taking an oral exam …the time it takes § But it is cheaper for a course of up to N students § What is N for your course? Do the math!

15 2016-09-23

Written- vs oral exam, teacher time

Written: Design and construction of exam and solution-sheet takes ≈___10-16 hours. Correcting one exam takes ≈___20 minutes

hours Oral: The exam takes ≈__30 minutes.

Moreover: Written (16 hours prep) Consider the Written (10 hours prep) gain at re-exam!

number of students

“We have 400 students in Introductory Physics… but we also have more than 10 professors who know the subject!”

16 2016-09-23

⏏

Master test

17 2016-09-23

The teaching trick: Do less of that which does not contribute

Spend less time (energy) on… listening to students complaints!

Before: What Professor V did: There were two individual The assignments were assignments in the course: renamed: • Homework 1 & 2 • MASTER TEST 1 & 2 The tasks were complex and (MÄSTARPROV) theoretical… Students complained bitterly What happened? and endlessly: • Complaints just stopped • The assignments come too • Students take the EARLY before we know how to assignments very do this! seriously – and are very • They are far too DIFFICULT and take TOO MUCH TIME! proud!

18 2016-09-23

…other interesting words… ⏏

Accident Evaluation Time out Certificate investigation Summit Grand challenge Jam session Weekly challenge Negotiation Dress rehearsal Dissection Show All hands on deck Opening Hackathon Master test Campaign Court hearing Talk show Demonstration Consultancy Stop-press Level up Gymkhana Pitch Workout Expert panel Show & Tell Elevator pitch Personal training Investigation Fair Pecha kucha Vernissage Workshop Keynote Speed dating Hearing Emergency room TED talk Match Review Launch Potluck Audition Test pilot Countdown Conference Ceremony Advisory group Pit stop Deadline Installation Working party Meeting Inspection Inauguration Q&A session

Fireworks

19 2016-09-23

The teaching trick: Do less of that which does not contribute (especially if it is expensive)

Spend less time on… writing feedback

Tax payer’s money down the drain!

Make the distinction between: § feedback for learning § justification of grade (does not generate learning, minimize cost)

20 2016-09-23

~ 40 students write an open-ended assignment of 4 pages (e.g. essay, design, reflection…)

§ The assignment is personal and important (a credo).

§ It would take several days to write good feedback!

§ Instead a final seminar - Intensive learning activity - Plenty of peer feedback and some from the teacher - Minimal summative assessment, sufficiently fair (pass/fail grade)

§ The teacher skims essays and makes quick decision: - Accepted to join the seminar - Pending acceptance, allowed to join but must submit improved version after the seminar (and they must tell the group and ask for guidance) - Reject, cannot join and must redo assignment the next time the course is given § Divides the students in groups of 4 (Usually one excellent essay, two medium good, and one needing improvement) § Sends mail with instructions - Download your colleagues’ work (from the digital platform). - Write ½ page constructive comments to each colleague, strong aspects and how the work can be improved. - Bring prints of comments to the seminar (4 for the group + 1 to the teacher). § This takes maximum 2 hours…

21 2016-09-23

Teacher prepares feedback before the seminar

§ Merges all essays into one big pdf. § Searches for a strong aspect in each text, making sure to cover the things that are important in the course. § Marks the passage with a ”star” in the margin with some keywords. § This takes just as long time as a hockey game J

[Recommending the GoodReader app for annotations]

At the seminar – group feedback § Discuss each essay with the aim to improve it (4*30 minutes). § Meanwhile, the teacher reads the written comments (to see that they were taken seriously + as input) § Their feedback is quite useful - Students are really good at pointing out deficiencies - Getting three different comments on your essay is great

22 2016-09-23

End with fireworks ⏏ 1 hour in plenary: § Display the pdf and discuss each ”Gold Star” full of enthusiasm and passion (fireworks). Bring it on! § End by recommending 3 – 4 essays to read before writing version 2.0 (for most students it is voluntary). § Publish the pdf in the digital platform as an invitation to browse.

Ultimate frisbee

23 2016-09-23

Dear Professor, ⏏

I coach the women’s ultimate frisbee teams and based on your workshop I changed our program for the practice weekend. Normally, since a game only involves 14 players, we would rotate and the others would do some drill on the side. Now, instead, I had a non-playing team standing on the sidelines and assigned each of them a player. Then I stopped the game periodically and had the sideline players give individual feedback to their assigned player. It went over remarkably well. A number of the ladies had very positive feedback, and said they had numerous strategy talks that they found incredibly helpful. It was also great for me, since I can’t possibly watch every player all the time. It was incredibly time efficient! So in conclusion, thanks again for the workshop. I thoroughly enjoyed it, and I thought you might like hearing about an application in a completely different “field”! Best regards, Professor D

The tricks are not only “alternative” teaching methods because the teacher is modern They address competences relevant for most educational programs. Make this explicit in the learning objectives!

After the course you should be able to (for instance) • evaluate your own work and the work by others… • critically analyse and give feedback on… • critically assess alternative solutions… • orally present and discuss your conclusions and the underpinning knowledge… • argue and contribute in discussions about…

Student: Why do I need to read their report? Teacher: Look at the course learning outcomes. This is how you practice to…!

24 2016-09-23

The tricks are not just “oil in the machinery”

More importantly they imply QUALITY TIME WITH YOUR STUDENTS - more meaningful, independent, value adding and fun!

Note: The most value-adding processes are often more stimulating The least value-adding processes are often boring routine tasks

25 2016-09-23

Also note that the most value-adding processes are the last to be replaced…

Still, it is not only a cliché that we only live once…

26 2016-09-23

The trick question

Do more of that which contributes to learning Easy part (especially when it is cheap)

Do less of that which does not contribute Hard part (especially when it is expensive)

Doing additional things on top of the old is not sustainable…

So why do we often keep doing things that are less effective for learning?

Discuss with your neighbours

What reasons can there be…? § Convenience – if I use traditional methods, there is no need to think, to make decisions, to explain, to defend, to persuade, to take responsibility… § It is correct: we actually never thought of this because we truly believed that it would always take more time… § Student expectations (or what we think they want…) § Colleagues expectations (or what we think they think…) § We teach in ways that make us feel good (lecture, have answers to everything, finish student work so it looks good…), without thinking so much about learning § We have not reflected on our routines and traditions § Lack of knowledge and fantasy in course design § We think education is more about sorting people than adding value § We actually think that everything is the students’ fault § Minimising risk: “when the old model doesn’t work, it is the student’s fault, but if I try something new and it doesn’t work, then it is all my fault”

27 2016-09-23

Implications for educational developers and researchers

§ It is great if teachers start analysing teaching from a learning perspective – even if they begin doing it for egoistic reasons. § To make change happen – and in particular to make it sustainable – we need to focus on how educational ideas can be implemented in reality. § Let us show what concrete practical instances of educational theory and philosophy would look like. § It is not sufficient to promote pedagogical ideas and theories on an abstract level only, focusing on the advantages for learning without tackling the issue of resource requirements. § Let us make realistic recommendations based on a better understanding and empathy with teachers’ work situations. § Let us understand what is blocking teaching innovation in the organisation, and what can help support innovation.

28 2016-09-28

Pick me!

The teaching trick: Do less of that which does not contribute

Spend less time on… ineffective tutorials.

1 2016-09-28

Student-Led Exercises Teacher preparations § Rename one tutorial per week as ”student-led recitation” § At course start, hand out problem sets with N problems, one sheet for each week Student preparations § Before each session, the students prepare to present their solutions on the whiteboard How the session works § When students arrive, they ”tick” on a list which problems they are prepared to present § Teacher ”randomly” picks a students to solve the first problem on the board § Discussion on alternative solutions, difficulties, ask the group to assist with problems - ”Did everyone solve it in the same way?” - ”I can see that only 8 of you ticked this problem, where did you others get stuck?” - ”Why was this problem different from the one last week, why couldn’t we use the same method?” § Pick a new student for the next problem, etc

Simple “rules”

Ticking a number of problems (e.g. 65%) is a course requirement. The student must The quality of the presentation is not assessed however demonstrate an and does not affect the grade since the purpose honest effort to prepare, and be able to lead a is purely formative. classroom discussion to a satisfactory treatment. If a student who is picked is obviously unprepared, all his/her ticks are removed for that recitation. This has luckily never happened!

2 2016-09-28

Results - Students

§ Students do better at the exam* Before: ~55% passed 2006 78 % Data from Per-Erik Hellström, 2007 70 % Semiconductor Devices, KTH, 2008 83 % 7.5 ECTS, undergrad year 2, 2009 86 % 25-30 students 2010 75 %

% passing exam

§ Student motivation increases due to: - Sessions are alive and fun - Lots of feedback and interactivity - Students are allowed to show what they can (to teacher and class) § Students like the format (4.2 on a scale 1 to 5)

Student experience - interviews How long time did you prepare for the student recitations? § I tried to do as many problems as possible, well all of them because it is good for the exam. (laughter) For each session… 6 hours maybe. (A) What was it like before the exercises? § We sat in a group and did the six problems, helping each other. Then the evening before I read through to get a good grip. Well, we sat… how long could it have been, 5 hours in the group and 2 hours on my own. (B) And when you study in groups, what is it you really do then? § We take a vacant classroom. Then you do one problem each and we stand together discussing it at the board. That’s how we do the problems, on the board. (B) If we look at the normal [teacher-led] exercises. § Oh, nothing at all, I just go there. You mean exercises where he solves problems, right? I don’t prepare for that, just copy the solution and try to follow. Student exercises are better because you have worked on the problems. You should do that in teacher-led sessions too, or at least read the problems. Then you would learn more. But you mostly copy the solutions. If you are lucky you understand. Otherwise it doesn’t give much. (B)

3 2016-09-28

Teacher experience “It is so much fun to discuss the subject with the students on a much, much, much higher level” Per-Erik Hellström KTH

Less work More effective as a teacher You have to design the assignments § See early and clearly what is difficult or before the course starts, but then: not • no preparation before the session § Also go faster when you know when • much fewer ”poor” exams to correct they are aboard § Learn to design problems that important, critical aspects

Analyse the activity: Why is the learning dramatically increased? Give the rationale for your answer. Note down your analysis and reflections on the A3 sheet. Reconvene

5 principles marked in Gibbs (1999) “Using Assessment Strategically…”:

1. Does the activity influence students to spend time-on- task? Does it also distribute this effort over time?

2. Does the activity generate appropriate learning activity?

3. Does it provide prompt feedback?

4. Does it provide feedback that the student pays attention to?

5. Does it help students internalise criteria for quality solutions and presentation?

4 2016-09-28

Analysis – Why is learning dramatically increased?

① Generates time on task! – Normally 6-7 hours study per week. – High attendance on recitations! – Punctuality. + Distributes study time during the course! – Makes all students study regularly from the first week.

Time on task - a word of warning § The aim is not to maximize time on task § The aim is to maximize learning

§ The teacher’s role is to help students spend sufficient time (easy to achieve) on appropriate tasks (this takes teaching skill).

[Analysis inspired by Gibbs (1999) Using Assessment Strategically to Change the Way Students Learn]

Analysis – Why is learning dramatically increased?

② Generates more appropriate learning activity! – Preparing the problems constitutes very good studies. – Further, it is not sufficient to arrive at the answer, they must also prepare to explain and present their solution. – Discussions give the whole answer.

Condition: Assignments worthy of this attention! § Assignments aligned with intended learning outcomes § They should illustrate critical and esssential aspects, reflect the desired understanding § Level of difficulty and complexity should be the same as in exam (good for student motivation) + New problems every time (dress them in slightly new clothes)

[Analysis inspired by Gibbs (1999) Using Assessment Strategically to Change the Way Students Learn]

5 2016-09-28

Quality of student learning (ii) Feisel-Schmitz Technical Taxonomy

Judge To be able to critically evaluate multiple solutions (bedöma) and select an optimum solution Solve Characterize, analyze, and synthesize to model a (lösa problem) system (provide appropriate assumptions) Explain Be able to state the process/outcome/concept in (förklara) their own words Compute Follow rules and procedures (lösa typtal) (substitute quantities correctly into equations and arrive at a correct result, ”plug & chug”) Define State the definition of the concept or is able to (återge) describe in a qualitative or quantitative manner

[Feisel, L.D., Teaching Students to Continue Their Education, Proceedings of the Frontiers in Education Conference, 1986.]

Constructive What should the students Formulating be able to do as a result alignment - of the course? intended applied learning outcomes

Constructive alignment Designing Designing activities assessment

What work is appropriate for What should the students do the students to do, to reach to demonstrate that they fulfil the learning outcomes? the learning outcomes?

6 2016-09-28

Analysis – Why is learning dramatically increased?

③ Generates prompt feedback! – Discussion is to give everyone feedback; they should go home with the whole answer. ⑤ Students develop a judgement for good solutions and good presentations! – They all see the variation…

The discussions should give the whole group feedback! § Get the students started discussing § Add any important aspects that needed to be addressed Good starters: - ”Did everyone solve it in the same way?” - ”I can see that only 8 of you ticked this problem, where did you others get stuck?” - ”Why was this problem different from the one last week, why couldn’t we use the same method?”

[Analysis inspired by Gibbs (1999) Using Assessment Strategically to Change the Way Students Learn]

Analysis – Why is learning dramatically increased?

④ Students care! – It creates motivation to expose and develop understanding together with the teacher and friends. – We see that students are eager to be picked (a chance not a risk).

Create a safe and friendly climate! § Never be rude or sarcastic to a student at the board (don’t let the students be either) § If you engage other teachers to run parallel groups, choose those who can also create a conducive atmosphere!

[Analysis inspired by Gibbs (1999) Using Assessment Strategically to Change the Way Students Learn]

7 2016-09-28

The Iceberg Principle

For each student-led presentation that we will see... up to 20 students have done the work to first solve the problem, and then prepare to present it.

”They are quite normal ”...what did it take, recitations where he five hours in the solves problems, right? I group and then two don’t prepare for that.” hours on my own.”

8 2016-09-28

⏏

What the professor said:

”It must be better that the teacher presents the problems, after all we are the experts and it is our job.”

⏏

The teacher’s fundamental task is to get students to engage in learning activities that are likely to result in their achieving the desired outcomes in a reasonably effective manner.

...remember that what the student does is actually more important in determining what is learned than what the teacher does.

[Shuell, quoted in Biggs 2003]

9 2016-09-28

⏏

”Teachers [are recommended to] embed useful study skills in their teaching so they are not just teaching what they want their students to learn, but how to learn it.” (Biggs, referring to Chalmers & Fuller 1996)

Said by a student in year 3 ⏏

There are in principle two kinds of courses in our program.

You have the unstructured ones where you don’t get any help with how or what to study. Like lectures and a thick book that you don’t know what to do with. There the exam is often very easy or they follow closely to old exams. Otherwise nobody would pass.

Then there are the structured ones where you have to work a lot during the course. But you know what to do and can concentrate on that. Even if you fall behind you know what to do to get back on track. There is less anxiety. You learn much deeper in those courses. But there is no chance to pass unless you really work hard.

10 2016-09-28

The CDIO approach for engineering education development

Kristina Edström KTH Royal Institute of Technology, Stockholm, Sweden

Kristina Edström

Engineer & Educaonal developer § M. Sc. in Engineering, Chalmers § Associate Professor in Engineering Educaon Development at KTH Royal Instute of Technology, Stockholm, Sweden § 700 parcipants in the 7.5 ECTS course Teaching and Learning in Higher Educaon, customized for KTH faculty, 2004-2012 § Director of Educaonal Development at Skolkovo Instute of Science and Technology, Moscow, 2012-2013

Strategic educaonal development, naonal and internaonal § CDIO Iniave for reform of engineering educaon since 2001 § SEFI Administrave Council, 2010-2013

Some publicaons § Crawley, E.F., Malmqvist, J., Östlund, S., Brodeur, D.R., and Edström, K. (2014) Rethinking Engineering Educaon: The CDIO Approach, 2nd ed., Springer Verlag § Edström, K., & Kolmos, A. (2014). PBL and CDIO: complementary models for engineering educaon development. European Journal of Engineering Educaon, 39(5), 539-555 § Edström, K. (2008) Doing course evaluaon as if learning maers most, Higher Educaon Research & Development, 27:2, 95 – 106

1 2016-09-28

What is CDIO? 1. An idea of what engineering students should learn and why “Engineers who can engineer”

2 2016-09-28

Stakeholder perspecves

Work life needs Employers Students

Society Faculty Engineering Educaon

An educaon about An educaon in engineering technology Conceive: customer needs, technology, enterprise strategy, regulaons; and conceptual, technical, and business plans Design: plans, drawings, and algorithms that describe what will be implemented Implement: transformaon of the design into the product, process, or system, including manufacturing, coding, tesng and validaon Operate: the implemented product or process delivering the intended value, NECESSARY BUT including maintaining, evolving and rering the system NOT SUFFICIENT

3 2016-09-28

Disciplinary theory Theory and judgement applied to applied to real problems “Problem-solving” Real problems § Cross disciplinary boundaries § Sit in contexts with societal and business aspects § Complex, ill-deﬁned and contain tensions § Need interpretaons and esmaons NECESSARY BUT (‘one right answer’ are excepons) NOT SUFFICIENT § Require systems view

Jonassen, D., Strobel, J., & Lee, C. B. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139.

Individual approach Communicave and collaborave approach § Crucial for all engineering work processes § Much more than working in project teams with well-deﬁned tasks § Engineering is a social acvity involving customers, suppliers, colleagues, cizens, authories, competors NECESSARY BUT § Networking within and across NOT SUFFICIENT organizaonal boundaries, over me, in a globalised world

4 2016-09-28

CDIO Standard 1: The context Educang for the context of engineering

Educaon set in Educate for the context of Engineering science Engineering

CDIO Standard 1 – The context Adopon of the principle that product, process, and system lifecycle development and deployment – Conceiving, Designing, Implemenng and Operang – are the context for engineering educaon. NECESSARY BUT Engineers who NOT SUFFICIENT can engineer!

But what if we do ask faculty?

Employers Students

Society Faculty Engineering Educaon

5 2016-09-28

Deeper working knowledge of disciplinary fundamentals

§ Funconal knowledge § Not just reproducon of didn’t known soluons to known problems ”get it” § Conceptual understanding § Being able to explain what they do and why

”got it” —

passed exam failed exam

See for instance Mazur, E. (1997) Peer Instruction, and Kember & McNaught (2007) Enhancing University Teaching.

Quality of student learning – more useful classiﬁcaons

Feisel-Schmitz Technical Taxonomy The SOLO Taxonomy Judge To be able to critically evaluate multiple solutions and select an optimum solution Solve Characterize, analyze, and synthesize to model a system (provide appropriate assumptions) Explain Be able to state the process/outcome/ concept in their own words Compute Follow rules and procedures (substitute quantities correctly into equations and arrive at a correct result, ”plug & chug”) Define State the definition of the concept or describe in a qualitative or quantitative manner

[Feisel, L.D., Teaching Students to Continue Their Education, Proceedings of the Frontiers in Education Conference, 1986.]

6 2016-09-28

What is CDIO? 2. A methodology for engineering educaon reform The 12 CDIO Standards

Success is never inherent in a method; it always depends on good implementation.

7 2016-09-28

The educaonal development process is the working deﬁnion of CDIO: The CDIO Standards Context: § Recognise that we educate for the pracce of engineering [1]

Curriculum development: § Formulate explicit program learning outcomes (including engineering skills) in dialogue with stakeholders [2] § Map out responsibilies to courses – negoate intended learning outcomes [3] § Evaluaon and connuous programme improvement [12]

Course development, discipline-led and project-based learning experiences: § Introducon to engineering [4] § Design-implement experiences and workspaces [5, 6] § Integrated learning experiences [7] § Acve and experienal learning [8] § Learning assessment [11]

Faculty development § Engineering skills [9] § Skills in teaching & learning , and assessment [10]

Crawley, et al (2007, 2014) Rethinking Engineering Education: The CDIO Approach, Springer.

CDIO Standard 2: Learning Outcomes Recognising the dual nature of learning

Understanding Professional of technical and engineering fundamentals skills

CDIO Standard 2 – Learning Outcomes Speciﬁc, detailed learning outcomes for personal and interpersonal skills, and product, process, and system building skills, as well as disciplinary knowledge, consistent with program goals and validated by program stakeholders.

8 2016-09-28

The CDIO Syllabus Support in formulang learning outcomes Each instuon formulates program goals considering their own stakeholder needs, naonal and instuonal context, level and scope of programs, subject area, etc The CDIO Syllabus § is not prescripve (not a CDIO Standard) § is oﬀered as an instrument for specifying local program goals by selecng topics and making appropriate addions in dialogue with stakeholders § lists and categorises desired qualies of engineering graduates § is based on stakeholder input and validaon

• Crawley, E. F. 2001. The CDIO Syllabus: A Statement of Goals for Undergraduate Engineering Education: see www.cdio.org/framework-benefits/cdio-syllabus-report • for version 2.0, see Crawley, Malmqvist, Lucas, and Brodeur. 2011. “The CDIO Syllabus v2.0. An Updated Statement of Goals for Engineering Education.” Proceedings of the 7th International CDIO Conference

National level learning outcomes For Master of Science in Engineering, students must demonstrate: Knowledge and understanding • knowledge of the scientific basis and proven experience of their chosen area of engineering, together with insight into current research and development work; and • both broad knowledge in their chosen area of engineering, including knowledge of mathematics and natural sciences, and substantially deeper knowledge in certain parts of the field. Skills and abilities • an ability, from a holistic perspective, to critically, independently and creatively identify, formulate and deal with complex issues, and to participate in research and development work so as to contribute to the development of knowledge; • an ability to create, analyse and critically evaluate different technical solutions; • an ability to plan and, using appropriate methods, carry out advanced tasks within specified parameters; • an ability to integrate knowledge critically and systematically and to model, simulate, predict and evaluate events even on the basis of limited information; • an ability to develop and design products, processes and systems taking into account people’s situations and needs and society’s objectives for economically, socially and ecologically sustainable development; • an ability to engage in teamwork and cooperation in groups of varying composition; and • an ability to clearly present and discuss their conclusions and the knowledge and arguments behind them, in dialogue with different groups, orally and in writing, in national and international contexts. Judgement and approach • an ability to make assessments, taking into account relevant scientific, social and ethical aspects, and demonstrate an awareness of ethical aspects of research and development work; • insight into the potential and limitations of technology, its role in society and people’s responsibility for its use, including social and economic aspects, as well as environmental and work environment aspects; and • an ability to identify their need of further knowledge and to continuously upgrade their capabilities.

9 2016-09-28

The strategy of CDIO is integrated learning of knowledge and skills !

Standard 3 – Integrated curiculum Integrang the two learning processes

The CDIO strategy is the integrated curriculum where knowledge & skills give each other meaning!

Acquisition of technical knowledge

CDIO Standard 3 – Integrated Curriculum A curriculum designed with mutually supporng disciplinary courses, with an explicit plan to integrate personal, interpersonal, and product, process, and system building skills.

Development of engineering skills

10 2016-09-28

Every learning experience sets 22 a balance and relaonship

Discipline-led learning Problem/pracce-led learning ¢ Well-structured knowledge base ¢ Integraon and applicaon, synthesis ¢ Open-ended problems, ambiguity, trade-oﬀs ¢ Evidence/theory, Model/reality ¢ Context ¢ Methods to further the knowledge froner ¢ Professional work processes CONNECTING WITH PROBLEM/PRACTICE ¢ ”Creang that which has never been” Ø Deep working understanding = ability to apply CONNECTING WITH DISCIPLINARY Ø Seeing the knowledge through the lense of KNOWLEDGE problems, interconnecng the disciplines Ø Discovering how the disciplinary knowledge is Ø Integrang skills, e.g. communicaon and useful collaboraon Ø Reinforcing disciplinary understanding Ø Movaonal context

Example: Communicaon skills in Lightweight design Communicaon in lightweight design means being able to § Use the technical concepts comfortably § Discuss a problem of different levels § Determine what factors are relevant to the situaon § Argue for, or against, conceptual ideas and soluons § Develop ideas through discussion and collaborave sketching § Explain technical maers to different audiences § Show confidence in expressing oneself within the field

The skills are embedded in, and inseparable from, students’ applicaon of technical knowledge.

The same interpretaon should be made for teamwork, problem solving, professional ethics, and other engineering skills.

”It’s about educang engineers who can actually engineer!”

11 2016-09-28

What does communicaon skills mean in the speciﬁc professional role or subject area?

[Barrie 2004]

Systemac assignment of programme learning outcomes to learning acvies - negoang the contribuon

Development routes (schematic) Year 1 Introductory course Physics Mathematics I

Mechanics I Mathematics II Numerical Methods

Year 2 Mechanics II Solid Product Mechanics development

Fluid Sound and Thermodynamics Mathematics III mechanics Vibrations

Signal Year 3 Control Theory Electrical Eng. Statistics analysis

Oral Written Project Teamwork communication communication management

12 2016-09-28

Engineering skills - implicaons

§ It’s not about ”soft skills” Personal, interpersonal, product, process, and system building skills are intrinsic to engineering and we should recognise them as engineering skills.

§ It’s not about “adding more content” Students must be given opportunities to develop communication skills, teamwork skills, etc. This is best achieved through practicing, reflecting, giving and receiving feedback (rather than lecturing on psychological and social theory).

§ It’s not about “wasting credits” When students practice engineering skills they apply and express their technical knowledge. As they expose their understanding among peers, doing well will also matter more to them. Students will develop deeper working knowledge.

§ It’s not about appending “skills modules” Personal, interpersonal, product, process, and system building skills must be practiced and assessed in the technical context, it cannot be done separately.

Place in Faculty perception of generic skills and attributes curriculum

Integral They are integral to disciplinary knowledge, infusing and ENABLING scholarly learning and knowledge.

Application They let students make use of or apply disciplinary knowledge, thus potentially changing and TRANSFORMING disciplinary knowledge through its application. Skills are closely related to, and parallel, discipline learning outcomes.

Associated They are useful additional skills that COMPLEMENT or round out discipline knowledge.They are part of the university syllabus but separate and secondary to discipline knowledge.

Not part of They are necessary basic PRECURSOR skills and abilities. We curriculum may need remedial teaching of such skills at university.

Barrie, S. (2004) A research-based approach to generic graduate attributes policy, Higher Education Research and Development. 23 (3), 261-275

13 2016-09-28

PROGRESSION

Enhancing progression through the curriculum

THE BLACK-BOX EXERCISE OUTPUT: Contribution to final learning outcomes INPUT: Course Input to later course Previous knowledge (black box) Input to later course and skills Input to later course

All faculty formulate their course only as input/output:

Input: “When students come to my course I want them to be able to…” Output: “When students leave my course they will be able to… because I think this is necessary input for course X…”

14 2016-09-28

Black-box exercise

All courses are presented through input and output only:

§ Enables efficient discussions § Makes connections visible (as well as lack thereof) § Gives all faculty an overview of the program § Serves as a basis for improving coordination § Use for adjusting intentions in planning phase § Use for checking existing programs

During the discussions: § Document which course takes responsibility for what learning outcomes § Identify redundancies or gaps § Check chronological order § Is it easy for the students to make the connections between courses?

Dimensions of progression

§ Subject content § Personal, professional and engineering skills § Theorecal maturity – not just ”more” theory, but Exercise for faculty: to make connecons and apply

(integraon, synthesis & modelling) • What important couplings between courses are already § Understanding context there and should be kept? (“real” problems, sustainable development, ethics, etc) § Selecng and applying methods, understanding • What important couplings limitaons between courses should be § Professional “eye” and language natural and obvious? (see and interpret situaons, discuss with others and relate to knowledge) § Academic wring, professional wring § Personal development (feedback, reﬂecon, etc) § View on knowledge (not just black and white) § Degree of independence as a learner (pedagogical red threads)

15 2016-09-28

Course Design for Integrated Learning

Learning What should the students be able to do as a result outcomes are Formulating of the course? the basis for intended course design learning outcomes

Constructive alignment [Biggs] Designing Designing activities assessment

What work is appropriate for What should the students do the students to do, to reach to demonstrate that they fulfil the learning outcomes? the learning outcomes?

16 2016-09-28

Construcve What should the students be able to do as a result alignment - applied Formulating of the course? intended learning outcomes

Designing Designing activities assessment

What work is appropriate for What should the students do the students to do, to reach to demonstrate that they fulfil the learning outcomes? the learning outcomes?

Construcve What should the students be able to do as a result alignment - applied Formulating of the course? intended learning outcomes

Designing Designing activities assessment

What work is appropriate for What should the students do the students to do, to reach to demonstrate that they fulfil the learning outcomes? the learning outcomes?

17 2016-09-28

Construcve What should the students be able to do as a result alignment - applied Formulating of the course? intended learning CDIO Standard 7 – Integrated Learning Experiences outcomes Integrated learning experiences that lead to the acquision of disciplinary knowledge, as well as personal and interpersonal skills, and product, process, and system building skills. Designing Designing activities assessment CDIO Standard 11 – Learning Assessment Assessment of student learning in CDIO Standard 8 – Acve Learning What work is appropriate for personal What should and interpersonal the students skills, do Teaching and learning based on acve and the students to do, to reach and product, process, and system to demonstrate that they fulfil building skills, as well as in experienal learning methods the learning outcomes? the learning outcomes? disciplinary knowledge.

Our curriculum system has 2 logical links

The strength of the chain – the extent to which graduates will actually meet the program learning objecves – hinges on:

§ the connecon between courses and programs that the sum of course learning objecves actually equals the program objecves, and § the construcve alignment that each course actually teaches and assesses students according to its learning objecves.

18 2016-09-28

Program learning objectives

Course learning Course objectives Course learning learning objectives objectives

Learning activities Assess- ment Assess- ment Learning Learning Assess- activities activities ment

Anyone can improve a course if it means that the teacher works 100 hours more

That is not a valid soluon… This is about how to get better student learning from the same (finite) teaching resources

CDIO Standard 10 -- Enhancement of Faculty Teaching Competence Acons that enhance faculty competence in providing integrated learning experiences, in using acve experienal learning methods, and in assessing student learning.

19 2016-09-28

Examples are illustraons of principles

A specific will example illustrate

generic to principles inspire

applications - of many different kinds.

Educational development in CDIO

Improving problem/pracce-based Improving discipline-led learning learning § Improving the quality of understanding § Adding problem/pracce-based § Knowledge prepared for use: seeing learning experiences the knowledge through the lense of – Early engineering experience problems – A sequence of Design-Implement § Ability to communicate and collaborate Experiences § Interconnecng the disciplines § Improving reﬂecon and learning § Improving cost-eﬀecveness of teaching

20 2016-09-28

A course in Basic Materials Science § Standard lecture based course § Focus on disciplinary knowledge (“content”) Hypoeutectoid steel was quenched from austenite to martensite which was tempered, spheroidized and hardened by dislocaon pinning..

[Professor Maria Knutson Wedel, Chalmers]

A course in Basic Materials Science

Two ways of seeing materials science From the inside - out From the outside - in “Materials engineers disnguish “Materials have a supporve role of themselves from mechanical engineers by materializing the design. The their focus on the internal structure and performance is of primary concern, processing of materials, speciﬁcally at the followed by consideraons of related micro- and nano-scale.” materials properes….” Flemings & Cahn Östberg

Performance Performance

Properes 500 nm Properes Manufacturing, Manufacturing processing Structure Material

[Professor Maria Knutson Wedel, Chalmers]

21 2016-09-28

A course in Basic Materials Science Implicaons I - formulang intended learning outcomes Old learning objecves New learning objecves (the disciplinary knowledge in itself) (performances of understanding) …describe crystal structures of some …select materials based on metals… consideraons for funconality and sustainability …interpret phase diagrams… ...explain how to opmize material …explain hardening mechanisms… dependent processes (eg casng, forming, joining) ...describe heat treatments… ...discuss challenges and trade-oﬀs when (new) materials are developed

...devise how to minimise failure in service (corrosion, creep, fractured welds)

[Professor Maria Knutson Wedel, Chalmers]

A course in Basic Materials Science Implicaons II - design of learning acvies Sll lectures and sll the same book, but And… framed diﬀerently: § Study visit in industry, § from product to atoms assessed by wrien § focus on engineering problems reﬂecon

§ Material selecon class (CES)

§ Acve lecturing: buzz groups, quizzes

§ Test yourself on the web

§ Students developed animaons to visualize

[Professor Maria Knutson Wedel, Chalmers]

22 2016-09-28

A course in Basic Materials Science Implicaons III - design of assessment 2011: New type of exam, aimed at deeper working understanding § More open-ended quesons - many soluons possible, the quality of reasoning is assessed § Interconnected knowledge – several aspects need to be integrated ØVery good results on the exam but some students were scared and there were many quesons beforehand…

2012: Added formave midterm exam, with peer assessment § Communicates expectaons on the required level and nature of understanding (Feedback / Feed forward) § Generates appropriate learning acvity § Early engagement in the basics of the course (a basis for further learning)

[Professor Maria Knutson Wedel, Chalmers]

Educational development in CDIO

In disciplinary courses In problem/pracce- § Improving the quality of understanding based courses § Knowledge prepared for use: seeing the § Adding problem/pracce-based learning knowledge through the lense of problems experiences § Ability to communicate and collaborate – Early engineering experience § Interconnecng the disciplines – A sequence of Design-Implement Experiences § Improving reﬂecon and learning § Improving cost-eﬀecveness of teaching

23 2016-09-28

Design-Implement Experiences student teams design and implement actual products, processes, or systems

§ Projects take different forms in various engineering fields CDIO Standard 5 – Design- § The essential aim is to learn through near- Implement Experiences authentic engineering tasks, working in A curriculum that includes two or more design-implement experiences, modes resembling professional practice including one at a basic level and one at an advanced level. § Progression in several dimensions Ø engineering knowledge (breadth and depth) Ø size of student teams Ø length of project Ø increasingly complex and open-ended problems Ø tensions, contextual factors Ø student and facilitator roles

The educaonal development process is the working deﬁnion of CDIO: The CDIO Standards Context: § Recognise that we educate for the pracce of engineering [1]

Faculty development § Engineering skills [9] § Skills in teaching & learning , and assessment [10]

Crawley, et al (2007, 2014) Rethinking Engineering Education: The CDIO Approach, Springer.

24 2016-09-28

CDIO integrated curriculum development - the process in a nutshell § Set program learning outcomes in dialogue with stakeholders § Design an integrated curriculum mapping out responsibilies to courses – negoate intended learning outcomes (both knowledge and engineering skills) § Create integrated learning experiences course development with construcve alignment ü mutually supporng subject courses ü applying acve learning methods ü an introductory course ü a sequence of design-implement experiences § Faculty development ü Engineering skills ü Skills in teaching, learning and assessment § Evaluaon and connuous improvement

What is CDIO?

3. A community to learn together and to share experience

The CDIO Iniave

25 2016-09-28

CDIO as a community – the CDIO Iniave

§ The CDIO Iniave started in 2000 as a project: Partners: MIT, KTH, Chalmers, Linköping University § Soon other instuons expressed an interest in joining, today more than 125 CDIO Collaborators worldwide

The international CDIO community Europe: § AFEKA Tel Aviv Academic College of Engineering North America Asia: § Astrakhan State University § Arizona State University § Beijing Institute of Petrochemical Technology § Bauman Moscow State Technical University § Beijing Jiaotong University § Cherepovets State University § California State University, Northridge § Chengdu University of Information Technology § Delft University of Technology § Daniel Webster College § Chulalongkorn University (Faculty of Engineering) § Duke University § Dalian Neusoft University of Information § Don State Technical University § Ernst-Abbe-University of Applied Sciences Jena § École Polytechnique de Montréal § Duy Tan University § Kanazawa Institute of Technology § Gdansk University of Technology § Embry-Riddle Aeronautical University § Kanazawa Technical College § Ghent University § LASPAU § Mongolian University of Science and Technology § Group T - International University College Leuven § Massachusetts Institute of Technology § Nanyang Polytechnic § Hague University of Applied Sciences § Naval Postgraduate School (U.S.) § Rajamangala University of Technology Thanyaburi § Helsinki Metropolia University of Applied Sciences (RMUTT) § Pennsylvania State University § Hochschule Wismar § Shantou University § Queen's University (Canada) § Instituto Superior de Engenharia do Porto § Singapore Polytechnic § Israel Institute for Empowering Ingenuity § Sheridan College § Suzhou Industrial Park Institute of Vocational § Kazan Federal University § Stanford University Technology § Taylor's University, School of Engineering § Lahti University of Applied Sciences § United States Naval Academy § Lapland University of Applied Sciences § Thu Dau Mot University § University of Arkansas § Tsinghua University § Moscow Aviation Institute § University of Calgary § Universiti Teknologi MARA (UiTM) § Moscow Institute of Physics and Technology § University of Colorado § Vietnam National University § National Research Nuclear University § Novia University of Applied Sciences § University of Manitoba § Yanshan University § Politecnico di Milano § University of Michigan Africa § Reykjavik University § University of Pretoria § University of Notre Dame § RWTH Aachen § ESPRIT, Tunisia Latin America § Saint Petersburg State University of Aerospace Instrumentation § Pontificia Universidad Javeriana UK-Ireland: § Aston University § Savonia University of Applied Sciences § School of Engineering of Antioquia (EIA) § Lancaster University § Technical University of Madrid § UNITEC Laureate International Universities § Queen's University (Belfast) § Seinäjoki University of Applied Sciences § Universidad Católica de la Santísima § South Eastern Regional College (SERC) § Siberian Federal University Concepción § Trinity College Dublin § Skolkovo Institute for Science and Technology § Universidad de Chile § University of Bristol § Telecom Bretagne § University of Leeds § Tomsk Polytechnic University § Universidad de Santiago de Chile § University of Leicester § Universidad del Quindio § University of Limerick § Tomsk State University of Control Systems and § Universidad del Quindío § University of Liverpool Radioelectronics (TUSUR) § Turku University of Applied Sciences § Universidad ICESI, Cali § University of Strathclyde § Universitat Politècnica de Catalunya § Universidad Nacional de Colombia, Bogota Sweden § University of Turku Australia § Chalmers § TU Madrid § AAEE § KTH § Ural Federal University § Linköping University § Chisholm Institute § Vilniaus Kolegija/University of Applied Sciences § Jönköping University § Curtin University § Østfold University College § Queensland University of Technology § Umeå University § Linnéaus University § RMIT § University of Auckland § University of Skövde § University of Sydney § Kristianstad University § University of the Sunshine Coast § Blekinge Institute of Technology § Luleå University of Technology

26 2016-09-28

Annual Internaonal Next: CDIO Conference § The Internaonal Fall meeng 2005 Queen’s University, Kingston, Canada November 2016, Porto, Portugal 2006 Linköping University, Linköping, § The European Regional meeng Sweden January 2017, Dublin, Ireland 2007 Hogeschool Gent, Gent, Belgium § 13th Internaonal CDIO Conference 2008 MIT, Cambridge MA, USA June 2017, Calgary 2009 Singapore Polytechnic, Singapore § 14th Internaonal CDIO Conference 2010 École Polytéchnique, Montreal, June 2017, Kanazawa, Japan Canada 2011 Denmark Technical University, Copenhagen, Denmark 2012 Queensland University of Technology, Brisbane, Australia 2013 Harvard/MIT, Cambridge MA, USA 2014 UPC, Barcelona, Spain 2015 CUIT, Chengdu, China 2016 Turun UAS, Turku, Finland

www.cdio.org

How to become a CDIO Collaborator 1. Express an interest (answer a few quesons) – Why does your university want to join the CDIO iniave? – Which of your programs do you plan to inially apply CDIO? How do you expect CDIO to influence these programs? – What goals do you hope to achieve? – What are your plans for parcipang with the other CDIO collaborang schools? – What experience do you have in engineering educaonal reform at your university, which might contribute to the effort and form a foundaon for the work as a collaborator? – What level of commitment and support do you have from your university's Dean and Central Leadership? – Who will be the key two to five parcipants in your effort? 2. Make introducons at a CDIO meeng 3. The CDIO Council will grant collaborator status

§ Contact the leader of your region, to get started. Juha Kono, Turku University of Applied Sciences. [Juha.Kono@turkuamk.ﬁ]

27 2016-09-28

What is CDIO? 1. An idea of what engineering students should learn: “Engineers who can engineer” 2. A methodology for engineering educaon reform: The twelve CDIO Standards 3. A community to learn and share the experience: The CDIO Iniave

Let us take a moment when everything is possible…

Focus on your program: What improvement would the program need? (5 minutes individually) What work would you want to engage in? (5 minutes with colleague)

Make small groups with common interests (20 minutes) § What important qualities are already in your programs and must be safeguarded? § What important qualities could be improved? § How can we work together?

28 2016-09-27

Integrated Learning in a Project Course

Jakob Kuttenkeuler, Naval Architecture Stefan Hallström, Lightweight Structures Kristina Edström

Jakob Kuenkeuler

§ Professor in Naval Architecture. § PhD in Aerospace engineering. § 10 years as director of two MSc programs and one PhD program. § Research on design process of high speed cra opmizaon for sustainability, Roung etc. § Teaches Hydrodynamics, Ship dynamics, Maneouvering, Propeller design, Sailing mechanics etc. § Awarded the KTH prize for outstanding educaonal achievements. § Engaged in CDIO since start.

1 2016-09-27

"Background"

Teaching vs learning Stakeholders needs/expectations Applications & skills I'm teaching...

Analysis & synthesis Engineering is fun! Authentic problem

Analysis Decisions Synthesis

Models (solving)

Some facts about the course

This project course

Thesis

Semester 2 Semester 3 Semester 4

§ 30-40 students in groups of 8-15 § 2 semesters, 20 ECTS (1/3 of students' time) § Individual grading A-F

§ 2 weekly scheduled hours but most activities "on demand” § Standard course funding (low material budget, limited teaching time) § Access to a standard classroom “owned” by the students (24/7) § Access to department workshops

2 2016-09-27

First day for each group of 10-17 students

Conceive, design, build and operate – a vehicle that can transport one person both at planing speed on water and at low speed submerged.

3 2016-09-27

How much should the success of the product influence the grades?

Interaction With Other Courses

This project course

Thesis

Semester 2 Semester 3

4 2016-09-27

Students create new things

• Teachers • Conceive • Open ended • Applied use of advise & coach, theoretical skills • Design • New year - new but not impose solutions group - new task • Whatever is • Implement designed has to • Neither students • Allow students • Operate be realised nor teachers know to grow into engineers the outcome in advance

From a distance, it looks like it is all about building cool products In fact – it is all about turning students into engineers!

5 2016-09-27

Always the same learning objectives

After the course the participant is expected to be able to: § analyse technical problems in a systems view § handle technical problems which are incompletely stated and subject to multiple constraints § develop strategies for systematic choice and use of available engineering methods and tools § make estimations and appreciate their value and limitations § make decisions based on acquired knowledge § pursue own ideas and realise them practically § assess quality of own work and work by others § work in a true project setting that effectively utilises available resources § explain mechanisms behind progress and difficulties in such a setting § communicate engineering – orally, in writing and graphically

The same learning outcomes are reached through different activities

Students do different tasks in the project a smörgåsbord syllabus for a smörgåsbord of students! § Presentations § Conceptual analysis § Experiments § "Expert" analysis § PR § Project management § Planning and follow-up § Manufacturing § …

➞ Students need to take individual responsibility for their learning outcomes

6 2016-09-27

Assessment challenges

• Individual grades (A-F)

• Assessing individual performance in a group setting

• Students work on many different tasks

• Teachers see only fragments of the actual performance

• Legal security / fairness

Assessment – the Introduction

Faculty • communicate course goals • instruct students to collect evidence in “portfolios”

Students • express personal individual goals • plan own activities

Start end

7 2016-09-27

Mid Course & Course End

Faculty • repeat course goals • discussion on giving/receiving feedback Students • write summary • read summaries, write feedback, suggest peer grades • read feedback & reflect • revisit/revise personal goals • follow-up on the process

formative summative

Start end

Summary: Sample (mid course) § L7. Effectively choose and use available engineering methods Status: Approaching. Ref: [4][5][6] I am trying but find it hard to find the balance between rough estimates and sophisticated computerized methods. Further, the word “effectively” does not apply on me. § L5. Make estimations, appreciating their value and limitations The propeller analysis required several estimations during its initial phase, e.g. the input power from the solar cells to the engine and the hull resistance. When working with the supporting structure for the hulls [72] the design loads acting on the craft were also approximated based on evaluation of the most critical loading conditions. These estimations were made in order to operate with some numbers and start the calculations. It was understood that having some, even rough, estimations will not let the process stop and will have only positive influence on the overall result. References: 1. Meeting minutes from … 2. Presentation, Preliminary design at design review #1 3. Experiment 4, Planning, execution and results 4. Report A 12, Hydrostatic stability - analysis 5. Report A107, Engine, design and mounting …

8 2016-09-27

What do you think, should the success of the product influence the grades?

Grades

The grades are set in relation to the intended learning outcomes based on a holistic assessment of:

• portfolios (reports, protocols, presentations, sketches, hardware, …)

• given feedback

• received feedback

• recommended grades from peers

• Participation, logged time and continuous observations by two teachers, independently

9 2016-09-27

What do you think - why is the assessment system so complicated?

What is the purpose of project work in educaon anyway?

Project goals

Project Learning objecves

10 2016-09-27

Powerful principle 1: the purpose is student learning

Powerful principle 2: Process for feedback and reﬂecon on experience

§ Teachers drive a process for rubbing students against each other

èbecause only reﬂecon can turn experience into learning

èfaculty role is to create and run a process - note the cost- eﬀecveness

11 2016-09-27

Powerful principle 3: Focus on process and individual - then the group and project take care of themselves

• Feedback is most effective for learning when it aims at students work processes and self-regulation, rather than the task at hand.

Hattie, J. & Timperley, H. (2007). The Power of Feedback.

Review of Educational Research, 77(1), 81–112.

• Individual grading because l Product grades are loosely coupled to learning outcomes – and create incentives not to learn

l Group grades create conflicts around ambition levels, invites free riders. These conflicts take focus from students, teachers and learning…

Powerful principle 4: Reversing the ’burden of proof’ § Each individual student is responsible for collecng and presenng evidence related to the learning outcomes (porolio)

èthis enhances reﬂecon and directs students aenon to the intended learning outcomes (- >learning)

èmakes the course format sustainable

12 2016-09-27

Powerful principle 5: ’for the good of the project’

§ The project and the group drives the speciﬁcaons, the needs, the deadlines... not the teachers!

èmakes everything students do in the course meaningful, reporng comes natural for the ﬁrst me

èmakes the course format sustainable

Maria Montessori: EVERY TIME YOU TIE THE SHOES FOR YOUR CHILD, YOU HINDER HER OWN DEVELOPMENT.

13 2016-09-27

Let’s hear some student voices

Interviews with students in the 2004 & 2005 cohorts (not the students in the picture...)

Interviewer: What did you learn about working in teams?

You knew theories before, empty phrases. But now I have seen them in reality. These things are so easy to say. Like [...]. I mean, you don’t have to be a rocket scienst to realise that, everyone knows it. But it’s one thing to know and another thing to apply, and we really got ﬁrst-hand experience from applying it. It is so obvious, you can stop anyone on the street and they would say ‘of course, everyone knows that’. But it is a completely diﬀerent thing to experience it in reality.

14 2016-09-27

Interviewer: So you chose not to switch project leader?

“[Changing the project leader] wouldn’t have furthered the project. It could only have suﬀered. But if you completely drop [consideraons for] the product - and maybe you should, actually – it might have furthered the course. It's hard to tell...you simply tend to put your focus on the product you are making.”

Tension between project and learning...

Interviewer: How do you think this course could be improved?

In the beginning I think there should have been some technical seminars to give a faster start of the project. Technical specialists who could have given a few lectures. To help you see possible designs for instance? Yes, technical soluons. And whom we could have contacted later with quesons. Hmm. I wonder if you may risk the main idea of the course? Yes... that is a risk... If they say ‘this is what you should do’... Yes, you are right. I can see that it’s been painful though. Yes, but maybe that’s what is good for us. But you think it would have been beer with a more eﬃcient start. Yes, but that is perhaps because it had led to a beer end result, I mean the boat. But maybe the learning wouldn't...

Tension between project and learning... Concepons of teacher’s and student’s roles are challenged...

15 2016-09-27

Interviewer: How do you think this course could be improved?

They should have been more like teachers. We had to do all the hard work ourselves and we don’t feel that we got as much help from the teachers as we could have had. [...] When we went and asked them ‘does this look alright’, they tried to answer as vaguely as they could. Just because they tried to make us solve things ourselves I think.

Student’s views on knowledge are challenged... Concepons of teacher’s and student’s roles are challenged...

Quote from a mid-course evaluaon

Not that these were the only calculaons needed, but the only ones that could be made. All the calculaons assuming kinemac equilibrium seem to give various degrees of unreasonable results. This is not just a pity and shame, but it is also terribly bad pedagogy now towards the end of an educaon. I would really have liked to see that the theory we have learnt was possible to use. We cannot even calculate the strength since everything is so ny.

Students with a black-and-white view on knowledge are seriously challenged...

16 2016-09-27

Student views must always be interpreted

• We noce that § students’ concepons of learning or atude towards knowledge is challenged § in students’ eyes, learning is oen overshadowed by the project per se

• The teacher will oen be blamed, as students think they should have been saved from the inconvenience.

• But these relevant challenges are not ”ﬂaws” that should be eliminated. They are key learning opportunies and we have no intenon to protect the students from them.

• It is then not appropriate to behave in conformity with student expectaons. But knowing they existed was valuable for course development.

• Conclusion: Don’t give the students what they want – give them something beer!

Powerful principle 1: the purpose is student learning § NOT reaching project goals (BUT the project sll drives learning and creates a movaonal context) § NOT technical sophiscaon (BUT there must be enough complexity and technical challenges to accommodate the learning outcomes) § NOT teacher popularity, or giving students what they want (BUT the students must sll have trust in the process and the teachers)

17 2016-09-27

The beauful sound of students growing into engineers... (I)

The greatest thing I have learned from this course is humility. I'll approach similar tasks more humbly in the future. We thought we were beer than we were. No, not beer, but we have taken courses with well-deﬁned problems, where there is an answer, the key. And that went well. But now you realised that as soon as you are confronted with reality, it’s quite another story.

The beauful sound of students growing into engineers... (II)

”It took some me (maybe even a month) before it felt like we really got started. We were fumbling around, doing tasks without really compleng them or seeing what was the conclusion, the next step from it. We wrote reports and said ‘we do this for our own sake’ but it took some me before that was actually the case. At least that’s how it was for me. But when that coin dropped, everything became very much easier.”

18 2016-09-27

...and more of the same...

“At the beginning of the course I was somewhat worried about ﬁnishing the educaon and starng to work as an engineer. Those worries are gone now. My conﬁdence in approaching technical problems and solving them has grown a lot.”

“Feedback was exchanged on everything between napkin scribbles at lunch to things you had built. This was valuable since it both gave me, and trained me to give, crique. It also helped me to see how other people are thinking and how they solve problems.”

“One of the best things during the project was that wrien documentaon was called for and that we in much lived up to those demands. It allows you to cross check things and check the work of yourself and others, and things are always available.”

CONSTRUCTIVE ALIGNMENT

What should the student Formulang be able to do as a result of the course? objecves

Designing Designing acvies assessment

What work should the What should the student do, to student do, to reach the demonstrate that they reached the objecves? objecves?

19 2016-09-27

REFERENCES

- Edström, El Gaidi, Hallström and Kuenkeuler (2005). Integrated assessment of disciplinary, personal and interpersonal skills - student percepons of a novel learning experience, Proceedings of the 13th Improving Student Learning, OCSLD, Oxford, UK. - Hallström, Kuenkeuler and Edström (2007). The route towards a sustainable design-implement course, Proceedings of the 3rd CDIO Conference, Cambridge, MA.

20 Technology Enhanced Learning (TEL) /E-learning

Johan Fridell E-learning Development Manager Business Solution Owner (BSO) e-learning [email protected] Technology for Teaching unit

• Education/courses within blended/hybrid/e-learning • Research within the domain of e-learning • E-learning system governance • E-learning development projects

As E-learning Development Manager I am responsible for our ability to understand and to meet demands from faculty when it comes to e-learning tools. E-learning is important to KTH

”The opportunities opened up by e-learning technologies have been incorporated, and the virtual campus is as important as its physical equivalent. Innovation in education shows a distinct link to technological and social innovation”

”Education at KTH is characterized by individualized learning in innovative learning environments”

Quotes from KTH Vision 2027 KTH definition of e-learning

”E-learning is defined as teaching with the support of technology. This includes using the Internet for learning activities and systems used for educational administration used by teachers”. Quote from KTH vision for e-learning KTH Vision for e-learning

KTH has a very clear vision for e-learning

KTH should use digital tools in all courses where it is deemed to lead to better learning

The aim is not to save time, but to use time in the best way possible

This vision does not exclude MOOC:s or distance learning, forefront courses/projects are necessary to test, study and develop new methods and tools. The best practices are then deployment for the whole university A perspective upon education in various formats.

Teacher/staff Participants

Campus

node

Research Education tools Methods and Company participants

Individuals An expanding set of tools and methods

TOOLS METHODS Lecture hall Lecture Projector Laboration Group room Seminar Video conference Workshop E-meeting software Flipped classroom Survey tool Peer review Learning Management Systems Blended Learning Video (streaming media) Net based course contents Books MOOC e-books … databases Collaboration tools Blogs Twitter Informal learning area Youtube … Guiding principle

A teacher and student centric governance model for IT Organisation of E-learning at KTH

System governance and development projects

Collaboration between ECE (business) and IT Department

“Förvaltningsobjekt E-lärande” LMS MOOCs

Course Programme Group Webs Bilda Webs Webs (KTH Social) (Ping Pong) Canvas edX (KTH Social) (KTH Social)

Moodle Artologik TurnItIn Urkund Kaltura Teacher support process

4. Support 1. Teacher 3. Support 5. Support 2.Support through tool support web throgth through via e-mail demonstrati (intranet) drop-in meeting/con on/workshop sultation Point of reference

KTH Teacher Survey 2013+2015

What do teachers at KTH use and what would they like to use? At KTH: Use (blue), Intererest in using (red)

)

tool tool info

Wiki

area tests

forum Blogs

check group review

clickers or reporting scanning

group evalutation Video clips Video course documents based conference

bokking Peer

( ind

result exam Groupmailings with

Web Web Discussion Plagiarism Video Video Project Course Shared meeting on computer Mentometers/ Paper Resource Curriculum on theCurriculum web Course E - Courseprogression Assignment webpage Public Top 6 functions (listed from highest)

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

)

tool tool area

( ind forum group

Blogs review check clickers

reporting scanning webpage

Video clips Video group evalutation documents

Peer bokking result exam Public Discussion Plagiarism Assignment Project Course Shared Mentometers/ meeting on computer Paper Resource Curriculum on theCurriculum web Course E - Courseprogression Current development within e-learning (Faculty demand driven) • Implementation of new LMS Canvas • Pilot courses autumn 2016 • Go live 2017 with several deep integrations

• Joining edX to offer Massive Open Online Courses • 2 courses running • 3 more in production • High ambitions for 2017 and 2018

• Deployment of videoplatform for all teachers • From pilot to enterprise implementation • Integration to Canvas

2016‐09‐29

KTH ROYAL INSTITUTE OF TECHNOLOGY Program Development and Management Hans Havtun

Program Director Energy and Environment [email protected]

Agenda

• The Energy and Environment program • The organization of the program • The program perspective • How students influence the program

1 2016‐09‐29

My background

• MSc Mechanical Engineering, KTH, 1995 • PhD Energy Technology, KTH, 2001 • Associate Professor in Energy Technology • Teaching at KTH since 1995, mainly Thermodynamics, Energy Utilisation, and Cooling of Electronics • Director of Studies, Dept Energy Technology, KTH, 2001-2009, 2011 • Educational Developer 2014-2016 • Program Director Energy and Environment since March 2016

The Energy and Environment program

-History - Program focus and Program outcomes - Courses at the BSc level - MSc programs available for the students

2 2016‐09‐29

History: Energy and Environment

• KTH’s newest 5 year engineering program (3 year BSc, and 2 year MSc) • Decision taken to start program in 2009 • Program developed during 2009-2010 (however, a lot of work had been done during 2008) • The first students were admitted at the fall semester 2010 • In 2015 the first students were graduated • So far, about 50 students have graduated from the program

Program focus – Sustainable development

• Sustainable development is by definition a cross- diciplinary subject area • The program attracts students with different interests and backgrounds • It offers a number of MSc programs from different schools at KTH • Courses are offered by five different schools

3 2016‐09‐29

Engineering degrees awarded Bologna convention Swedish convention

BSc 3 years MSc in Academic Professional Engineering 5 years degrees degree

MSc 2 years

Even though the courses may be identical, the program outcomes differs slightly

Program outcomes (5 year program)

In addition to the objectives specified in the Swedish Higher Education Ordinance, a graduate Master of Science in Engineering from Energy and Environment at KTH shall …

4 2016‐09‐29

Program outcomes Knowledge and understanding • have basic knowledge of all aspects of the energy system in a broad sense, which includes the technologies and subsystems that are found in all stages from energy source to the energy's end use, and be able to understand these as socio-technical systems consisting of both technical components and the actors that develop, manages and use the system

• have good knowledge of the processes of modelling, simulation and validation of energy and environmental systems using modern engineering tools

• possess good knowledge of conditions relating to innovation, corporate enterprises and business in terms of the planning, strategies and objectives of businesses within the energy and environment sector

Program outcomes Skills and abilities • be able to describe sustainable development and relevant environmental problems at a foundational level, i.e., visions, concepts, definitions, and be able to provide a description of the current global situation • be able to, in a professional way, express themselves and communicate thoughts, ideas, visions and results to those in their professional proximity and the surrounding community • be able to critically analyse the historical and future importance of the energy and environment sector for global and local societal development and its relation to ecological systems • be able to compare and discuss different perspectives on issues of importance to sustainable development

5 2016‐09‐29

Program outcomes Ability to make judgments and adopt a standpoint • have a holistic view of sustainable development with systems and life-cycle thinking for products and services and for technical systems, based on an interdisciplinary approach and based on different actor perspectives • have the ability to assess ethical issues and conflicts of objectives relating to sustainable development, and demonstrate a deep knowledge of the engineer's role and responsibilities in society, especially regarding social and economic aspects and environmental/ecological aspects • have the skills to challenge, develop and problematise prevailing habits, thought patterns, technical and economic systems, and cultural and societal values.

Courses at the BSc level

• At the BSc level, the program has 19 compulsory courses, • 3-4 conditionally elective prerequisite courses, and • 1 freely elective course

• The conditionally elective prerequisite courses are chosen based on the MSc program the student want to attend

6 2016‐09‐29

Year 1

P1 P2 P3 P4

Energy, climate, Ecology and Mechanics Basic and environment Environmental Chemistry Effects

Algebra and Calculus in One Calculus in Several Electromagnetism Geometry Variable Variables and Waves

Year 2

P1 P2 P3 P4 Numerical Methods and Basic Programming Material and Environmental Energy Balances Systems Analysis Probability Theory Differential and Statistics Equations Electrical Circuit Energy Systems Thermodynamics Analysis

7 2016‐09‐29

Year 3

P1 P2 P3 P4

Energy Systems in Environmental Society Economics

Bachelor Thesis

The empty spaces are filled with conditionally elective prerequisite courses decreed by the MSc programs, and one freely elective course

Year 3 – Electric Power Systems

P1 P2 P3 P4

Vector Analysis Energy Systems in Environmental Language Society Economics Course Electromagnetic Theory, introduction course Bachelor Thesis Automatic Electric Power Control Systems

8 2016‐09‐29

MSc programs available • Electric Power Engineering • Sustainable Energy Engineering At KTH • Sustainable Urban Planning and Design • Chemical Engineering • Environmental Engineering and Sustainable Infrastructure • Sustainable Technology At least • Environomical Pathways for Sustainable Energy Systems one • Renewable Energy semester • Smart Electrical Networks and Systems at a foreign • Energy for Smart Cities university*

* Through KIC Innoenergy

Positions of graduated students Energy consultant, private sector Energy and climate advisor, municipality Project engineer, government office Building project manager, private sector Land management engineer, private sector Surface water and sewage water project manager, private sector Electric supply network investigations manager, private sector Project manager, Energy in buildings, private sector Technical project manager, private sector Waste manager, municipality

9 2016‐09‐29

Agenda

• The Energy and Environment program • The organization of the program • The program perspective • How students influence the program

The organization of the program

- Steering group (Director of undergraduate education of school) - Program management group Steering - Program development group Group - Sustainable development group Program management group

Program Sustainable development development group group

10 2016‐09‐29

Steering group As the program is multi-disciplinary, a steering group consisting of representatives from the four schools at KTH with interest in the program:

Industrial Engineering and Management Architecture and Built Environment Electrical Engineering Chemical Engineering

Decides on economical issues and strategical changes in the program

Program management group

Program director Program secretary Student counsellor International coordinator

• Daily program management, handle exchange students • ”Planning” courses that are in the program • Responsible for fulfillment of program outcomes • Responsible for approving diploma applications

11 2016‐09‐29

Program development group

Consists of: Program management group Master program directors Teachers representing courses with special roles in the program Student representatives

• Suggest/Investigate changes to the program => Steering group • Meets 3-4 times a year

Sustainable development group

Consists of: Vice program director (responsible for sustainable development) Representatives for each master program Student representatives

• Suggest changes in courses/program to enhance sustainable development aspects • Assurance that sustainable development outcomes in the program are met

12 2016‐09‐29

Agenda

• The Energy and Environment program • The organization of the program • The program perspective • How students influence the program

The program perspective

A number of Courses builds the Program How can the Program director influence courses? A course in the program may be offered: - by a different school! - to several programs with different program outcomes! Quite hard to - influence courses as a program director - assess fulfillment of program outcomes

13 2016‐09‐29

How to assess fulfillment of program outcomes

Each program outcome is breaken down into smaller sub- outcomes All courses are investigated to see which program outcomes are fulfilled and to what degree (partial, full) This is done by looking at the assessment of course modules and the course learning outcomes linked to them Finally, a table of all the program outcomes, sub-outcomes, courses, and course modules that fulfills program outcomes can be compiled

Example

Program outcome: have basic knowledge of all aspects of the energy system in a broad sense, which includes the technologies and subsystems that are found in all stages from energy source to the energy's end use, and be able to understand these as socio-technical systems consisting of both technical components and the actors that develop, manages and use the system

14 2016‐09‐29

Example

Discerning sub-outcomes: •… • … technologies and subsystems that are found in all stages from energy source to the energy's end use … •….

Example

Program sub-outcomes: have basic knowledge of… technologies and subsystems … • Energy sources • Energy conversion • Energy end-use

15 2016‐09‐29

Example

Course: Thermodynamics Course Module(s): Examination, Assignments Course outcomes: After the course, the student should be able to: • formulate, model, and solve problems involving systems and devices having various forms of energy exchange and energy conversion. • model systems, and to be able to identify sub-systems and components in engineering systems. • present stringent and understandable solutions to problems in the field of thermodynamics.

Example Apparently, the Thermodynamics course contributes to the program outcomes. To what degree? Is the program outcome fully met? As it does not cover all types of energy conversion, the program outcome is partially fulfilled.

Program Sub-outcome Course Assessment Degree outcome module 1 Energy Thermo- Examination, Partial conversion dynamics Assignments 1 Energy Energy Project Partial conversion Systems

16 2016‐09‐29

Agenda

• The Energy and Environment program • The organization of the program • The program perspective • How students influence the program

How students influence the program

Students play a very important role in program development • They are represented in (almost) every deciding body at KTH • They provide feedback to courses and the program • They are represented in the student union • They arrange their own program evaluation day

17 2016‐09‐29

How students influence the program The student union appoints representatives that sit in the KTH board. The students in the Energy and Environment program have their own branch of the student union. The branch has a studies committee that monitor the quality of the program. The committee has one chairperson, and a vice chairperson. Each program also has a program responsible student. Each class has student representatives. These representatives attend the meetings arranged by the program: Program conference, Program development group, Sustainable development group, Schedule planning meeting, Semester start-up meeting, ”Link meetings”

Support activities for the program

PD network: All program directors at KTH meet once a month Program conference: Held every year where all teachers in the program and student representatives meet to discuss program development. Schedule planning meetings: teachers having parallel courses meet prior to scheduling their courses to avoid clashes of exams, deadlines, etc. Semester start-up meetings: teachers having parallel courses meet a week before the semester starts to inform each other about deadlines in their respective courses. ”Link meetings”: Teachers having parallel courses and student representatives meet twice during each semester to discuss progress of the semester Program development + Sustainable development group meetings

18 2016‐09‐29

Ongoing program work

• Revision of break-down of program outcomes into sub- outcomes • Visiting MSc programs to ensure that the program outcomes are met - all specializations of all MSc (Difficult as the MSc program also is connected to other engineering programs). - progression of skills and abilities - sustainable development outcomes • Formation an industrial reference group