D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning

T-FLIP D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning

Førsteamanuensis, Morten Goodwin, Ph.D. 2015-11-30

T-FLIP D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning

Førsteamanuensis, Morten Goodwin, Ph.D.

Universitetet i Agder

2015-11-30 T-FLIP

• T-FLIP: Testdrevet Ferdighetslæring i Programmering • Large diverse student groups in programming. • All need individual attention and motivation. • Should learn what the industry needs.

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 2 T-FLIP Ambitions

WP2: Industry-near testing WP4: Learning based on needs

WP5: Peer review WP3: Motivation and gamiﬁcation

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 3 T-FLIP Ambitions

WP2: Industry-near testing

• Automatic testing. • Industry tools. • Industry partners: Skatteetaten, Lærelyst

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 4 T-FLIP Ambitions

WP4: Learning based on needs

• Automated testing. • Tests tailored for students. • Flexible automated feedback. • Active students.

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 5 T-FLIP Ambitions

• Review of each other. • “Best” students review-responsible

WP5: Peer review

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 6 T-FLIP Ambitions

• Practice as much as possible? • Instead of FAIL, give hints, videos, URLs. • Start with easy tasks, and “earn” more difficult tasks. • Assignments that never finish. WP3: Motivation and gamification

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 7 Projects

• Fjelltools: Completed • TCLT: Ends 2015-12.

• Autograder

• T-ﬂip 2015-2017. • Largest project funded from 2015. • 1/6 of their budget. • Total budget: 3 MNOK (incl. oc) • Partners: • University of Agder, University of Stavanger, Oslo and Akershus University College, Noroﬀ Unive, Lærelyst T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 8 →

↑ ↓

←

Old Way

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 9 ↑ ↓

←

Old Way

→

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 9 ↑

←

Old Way

→

↓

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 9 ↑

Old Way

→

↓

←

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 9 Old Way

→

↑ ↓

←

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 9 T-FLIP

T-FLIP

Teaching Material

Industry Near Industry Near Teacher Assignments Assignments

Code Student Version Control Automated Pass Feedback Pass Fail

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 10 Student Handins

→ → Combined GIT

Teacher Tests

↑ ↓

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

Number of parts approved: 1 av 4 (25%) Total assignment 13: Not approved

T-FLIP Way

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 11 Student Handins

→ Combined GIT

Teacher Tests

↑ ↓

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

Number of parts approved: 1 av 4 (25%) Total assignment 13: Not approved

T-FLIP Way

→

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 11 ↑ ↓

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

Number of parts approved: 1 av 4 (25%) Total assignment 13: Not approved

T-FLIP Way

Student Handins

→ → Combined GIT

Teacher Tests

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 11 ↑

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

Number of parts approved: 1 av 4 (25%) Total assignment 13: Not approved

T-FLIP Way

Student Handins

→ → Combined GIT

Teacher Tests

↓

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 11 ↑

T-FLIP Way

Student Handins

→ → Combined GIT

Teacher Tests

↓

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

NumberT-FLIP of parts approved - D-TEL2015: 1 av 4 (25%) Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Total Goodwinassignment 13: Not approved 11 T-FLIP Way

Student Handins

→ → Combined GIT

Teacher Tests

↑ ↓

Assignments 1 2 3 4 5 6 7 8 9 10 11 12 13

Assignment 13 Part 1: Approved

Part 2: Approved ←

Part 3: Approved

Part 4: Not approved Tried to click ”restart”, but original game was not restored.

Student Joint Repository Assignment Repository Repository

Hand-ins Assignments User interface

Pass Pass Fail

Student Teacher

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 12 Usage - DAT101

45% 40% 35% 30% 25% 20% 15% 10%

5% New Assignment Given Deadline 0% Wed Thu Fri Sat Sun Mon Tue

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 13 Usage - DAT101

2.2

1.8

1.6

Commits per studebnt 1.4

1.2

1 1 2 3 4 5 6 7 8 9 Assignment • Anomolies removed: The week the system went down and we had to evaluate manually.

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 14 Courses involved

• University of Agder: • DAT101: Introduction to Programming • DAT220: Advanced Programming • DAT111: Basic C Programming • DAT234: Scripting and Hacking (to some extent) • DAT201: Algorithms and Data Structures (planned) • University of Stavanger: • Distributed Systems • Introduction to Programming (planned) • Oslo and Akershus University College of Applied Sciences • Scripting (planned) • Noroﬀ University College and Noroﬀ Vocational School: • Basic C-programming (planned)

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 15 Distributed Systems: University of Stavanger

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 16 Introduction to Programming: Noroﬀ University College

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 17 Future plans: Adaptive Learning

Student A Student B Student C Experienced at Experienced at Finds the logical programming. programming. programming Finished with the Finished with the assignments easy. assignments early. assignments early. Needs repetitive Not challenged Not challenged practice with object enough. enough. orientation.

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 18 Future plans: Peer Review

Program/correct the assignment

Deliver assignment to Peer Review Feedback/learning from Peer Review

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 19 Conclusion

Students are working more

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 20 Conclusion

Morten is working less

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 21 Conclusion

Everyone is happy

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 22 T-FLIP - Group

T-FLIP - D-TEL2015 Symposium on Advances in Digital Technologies for University Teaching & Learning - Morten Goodwin 23