Masters in Data Science

M.Eng (Structured): Industrial Engineering Focus: Data Science

Host: Department of Industrial Engineering (in collaboration with other Engineering Departments, Applied Mathematics and Computer Science) Stellenbosch University

1

in early detection of changes in the production system Introduction performance and minimization of expensive wasted production time. Data science (DS) is the scientific investigation that 4. Supply chain performance prediction employs innovative approaches and algorithms, most notably machine learning algorithms, for processing and The process of delivering a product from customer order analysing data. DS technologies can be applied to both to order delivery involves a large number of resources small and big data, of various types such as relational, and incorporates a large amount of uncertainty. Different images, video, audio, and text. Big data constitutes supply chain partners e.g. warehouses, transport services, extremely large data sets that may be analysed manufacturers computationally to reveal patterns, trends and and more may associations, especially relating to human behaviour and be involved in a interactions. specific order. DS techniques This programme focuses on enabling students to develop can add immense innovative optimisation and machine learning techniques value in making to produce novel, efficient and robust data science sense of the technologies, for use in Industrial Engineering, Engineering millions of Management and related applications. records of Examples include: supply chain data 1. Forecasting generated on a Forecasting customer daily basis. demand requires the 5. Process Monitoring analysis of a large Advances in online monitoring and data collection present amount of data and is an opportunity to enhance the efficiency, sustainability essential to operating and profitability of chemical and mineral processing plants. an efficient business For example, condition-based maintenance is an approach which meets to monitor the condition of assets and inform decisions customer regarding planned preventative maintenance; machine requirements. learning techniques can be used to provide deeper insights Retailers conduct into equipment conditions using all available millions of measurements. Another example includes operation state transactions on a identification, whereby normal operating conditions daily basis generating (NOC) are defined based on feature extraction from datasets which suppliers then can utilize to understand measured data. Any deviation from NOC can be used to demand patterns and schedule production and deliveries. identify potential faults, and the origin of the fault can be 2. Customer segmentation and targeted traced using causality analysis. These are but a few of the marketing machine learning techniques currently applied in the A key concept in the business world is that not all process engineering industry. customers are the same. Different customers may have a different impact on a company’s profitability and each Structured Program Contents customer will most often demonstrate unique buying behaviour. DS techniques can play a significant role in The program consists of 8 x 15 credit modules, that will determining the optimal segmentation of customers, be presented in blocks (each block has a duration of 1 where companies often have hundreds of customers week), and students must attend these blocks at which need to be serviced. Furthermore, customer order Stellenbosch University. A project in each module will history can be used to predict future behaviour and then be required to test the application of the theory develop targeted marketing strategies. exposed to in the module. 3. Production quality control A final project must then be completed, where the knowledge gained in all 8 modules can be applied on a A large number of quality control and assurance systems relevant industry related project. have been developed over the last couple of decades to monitor production and predict when a production The detailed course content is as follows: system will no longer meet specifications. The use of more intelligent DS algorithms can play a significant role

2

Module Description Credits Responsible Generic Structured Masters Modules: Students need to select 2 out of the 4 modules below Project The module focuses on advanced topics in project management, and it is expected 15 Industrial Management that participants have either attended a project management course or have Engineering 51993-873 experience in managing projects. The module builds on the traditional project scheduling by addressing critical chain management and looks at managing project risks through the identification and assessment of risk potentials and mitigating strategies, including resource / cost management and contingency planning. The selection of appropriate teams and structures to facilitate contract management are discussed, along with executing project leadership through proper communication channels. The importance of procurement, from tender procedures through to supplier selection will be highlighted. The different nuances between commercial and research projects will be explained. Advanced Topics in The purpose of the module is to present principles of general management within 15 Industrial Engineering the context of technical disciplines. The course themes include the business Engineering Management environment and strategic management on a firm level, touching on the role of 11478-873 innovation and technology for competitiveness on a systems level from international and national perspectives. The course will include a significant focus on tools and techniques for technology and innovation management exploring the link between technology management and business management taking a capabilities approach. These capabilities include acquisition, protection, exploitation, identification and selection. We relate traditional approaches to technology management to what it means for the context of the fourth industrial revolution, platform economies and innovation platforms. The functions of engineering management, namely planning, organising, leading and controlling will also be discussed. This will include a specific focus on human resource management, both insofar as managing projects, people and groups is concerned as well as aspects of labour relations and specifically the labour law and contractual requirements in South Africa. We contextualise the above under the theme of “leadership”, with an exploration of different leadership styles, communication and motivation. Numerical Methods The module focuses on matrix computations. We study the effective solution of 15 Applied TW876 linear systems, involving both square and rectangular matrices (least-squares). Mathematics Direct as well as iterative methods are considered, with the emphasis on sparse matrices and matrices with structure. Numerical methods for the eigenvalue problem are also considered. Pitfalls such as numerical instability and ill-conditioning are pointed out. Model problems are taken from partial differential equations, data analysis and image processing. Theory, algorithmic aspects, and applications are emphasized in equal parts. Project Finance The module focuses on how to finance a business opportunity (project) that can be 15 Civil 58157-812 isolated from the rest of a company’s business activities. Financing through a Engineering combination of debt and equity are discussed, based on the future profitability of the project where project cash flow is the main source of capital recovery and the project assets are the only collateral. The concepts of construction loans and public- private partnerships are discussed. A number of case studies will be covered in the module. Module content: • Infrastructure and development finance: Sources of business finance and private sector project financing models. • Review of time value of money / discounted cash flow / interest calculations. • Basic accounting statements (balance sheet, income and cash flow statements. • Costing and management accounting – theory / techniques and costing system concepts. • Ratio analysis, from basic ratios to the DuPont approach. • Economic analysis of investment decisions. • Market valuation (EVA and MVA). • Value drivers in the company, sustainability and the Balanced Scorecard. • The national accounts and economic growth. • Feasibility studies and techno economic analysis: • System identification, parameter identification, environment and system boundary • Definition, environmental scanning, system modelling and simulation concepts, modelling • Risk and uncertainty in infrastructure finance and project development.

3 Module Description Credits Responsible • Materials, labour and equipment: Impact of required service and quality levels. Cost estimation and cost controls of construction projects. • Revenue stream estimating and modelling. Financing models. Compulsory Modules Data Science Data science is the application of computational, statistical, and machine learning 15 Jacomine techniques to gain insight into real world problems. The main focus of this module is on the data science project life cycle, specifically to gain a clear understanding of the five steps in the data science process, namely obtain, scrub/wrangling, explore, model, and interpret. Each of these steps will be studied with the main purpose to gain an understanding of the requirements, complexities, and tools to apply to each of these life cycle steps. Students will understand the process of constructing a data pipeline, from raw data to knowledge. Case studies from the engineering domain will be used to explore each of these steps. Applied Machine In this module students will be exposed to a wide range of machine learning 15 Andries Learning techniques and gain practical experience in implementing them. Students will not Thorsten only learn the theoretical underpinnings of several machine learning techniques, gaining an important understanding of the requirements, inductive bias, advantages and disadvantages, but also will gain the practical know-how needed to apply these techniques to real-world problems. The focus will be on information-based learning, similarity-based learning, error-based learning, kernel-based learning, probabilistic learning, ensemble learning, and incremental learning. Big Data This module focuses on the tools and platforms for big data management and 15 External Technologies processing. Big data management refers to the governance, administration and Specialist organization of large volumes of data of different types (both structured and unstructured). Efficient platforms to store and manage big data will be considered, including NoSQL, data warehousing, and distributed systems. Big data processing focuses on the 3V-characteristics of big data namely volume, velocity, and variety. Different architectures for big data processing will be studied, including map-reduce and graphical big data models. Students will obtain experience in big data tools and platforms, including Spark, Hadoop, R, and data virtualization. Other aspects of big data, such as data streams, data fusion, and data sources, including social media and sensor data, will be discussed. Data Analytics for In this module students will learn the data analytics life cycle, and how to apply each 15 Andries Engineers phase of this life cycle to solve engineering data analytics problems. Students will learn techniques for exploratory data analysis, and how to apply machine learning approaches for mining knowledge from data sets, to extract hidden patterns, associations and correlations from data. Students will gain the practical know-how needed to apply data analytics techniques to structured data. Students will learn advanced approaches to data analytics, with a specific focus on visual analytics, image analytics, text analytics, and time series analytics. The student will gain experience in the implementation of various techniques to extract meaning from these different data source types. The advanced data analytics techniques encountered will be applied to data intensive engineering problems. Optimisation for In this module students will learn about different classes of optimisation problems 15 Jan Data Science that can occur in the engineering domain, and will learn how to characterise the complexities of these optimisation problems. The student will learn a wide range of advanced meta-heuristics and hyper-heuristics that can be used to solve these different classes of optimisation problems. The student will gain experience in implementing advanced optimisation algorithms to solve real-world engineering optimisation problems. As one of the application areas, the module will explore ways in which optimisation techniques can be applied to improve the performance of machine learning algorithms, and to easily adapt machine learning approaches to non-stationary environments and data streams. Selected Specialization Module – Students need to select one of the choices below: The availability and time schedules of these courses must be confirmed before enrolment. (Some of the modules may be semester modules) Advanced Design This course aims to expose students to the solution of optimization problems in 15 Mechanical and 814 engineering. The course will follow a systematic approach to solve optimization Mechatronic problems: Engineering • identify optimization problems in engineering, • construct mathematical programming problems, • select an appropriate optimization strategy, • obtain a solution to the mathematical programming problem, and • interpret the solution of the mathematical programming problem.

4 Module Description Credits Responsible Robotics 814 Mathematical modelling of robots; Rigid motions and homogeneous 15 Mechanical and transformations; Forward and inverse kinematics; Denevit-Hartenberg convention; Mechatronic Velocity kinematics: the Jacobian, singularities; Path and trajectory planning; Engineering Independent joint control; Robot dynamics: Euler-Lagrange equations, kinetic and potential energy, equations of motion, properties of robot dynamic equations, Newton-Euler formulation; Force control; Computer Vision: camera calibration, image segmentation, vision and servo control. Advanced Control Contents: This module is concerned with control systems design and analysis for 15 Mechanical and Systems 814 MIMO (multi-input-multi-output) systems with uncertainties. It covers basic linear Mechatronic algebra, block diagram algebra for MIMO systems, loop shaping analysis and design, Engineering internal stability, generalized Nyquist stability criterion, all stabilizing controllers, MIMO robustness, generalized plant, linear fractional transformation (LFT), nominal/robust stability/performance (NS, NP, RS, RP), representing uncertainties, optimal control (LQ, Kalman filter and LQG), and H-infinity optimal control. Advanced Dynamics Formulate and solve the dynamics of a particle or system of particles: Relative to 15 Mechanical and 814 static or moving axis system; in terms of generalized coordinates and constraints; Mechatronic in terms of virtual displacement and work; in terms of the Lagrange and Hamilton Engineering energy principles; for impulsive forces. Formulate and solve the kinematics and dynamics of a rigid body: In terms of rotation kinematics; with the modified Euler rotation equations of motion; for impulsive forces and moments. Foundations of Many view the advent of deep learning as a revolution that has fundamentally 15 Applied Maths Deep Learning transformed modern ML and AI. This module will cover the basics of deep learning as a precursor for more advanced modules in the MSc programme. It will start with a quick recap of ML fundamentals, namely training, generalisation, overfitting, cross- validation, regularisation, and hyper-parameter optimisation. The following topics specific to neural networks will then be covered: multi-layer perceptrons, deep feedforward neural networks, gradient-based training and backpropagation, convolutional neural networks, recurrent neural networks, attention mechanisms, autoencoders and deep generative models. Probabilistic Probabilistic modelling and reasoning form a cornerstone of modern ML and AI. 15 Applied Maths Modelling and The module will recap relevant concepts from Probability Theory, including Bayes’ Reasoning theorem and conditional independence. It will then cover marginalisation, sum- product decomposition, Markov blankets, classic hidden Markov models, expectation-maximisation, probabilistic graphical models and data completion. The module will also cover basic information theory, including entropy, mutual information and Shannon’s theorem. Applied Machine ML and AI drive the back-ends and front-ends of many large online companies and 15 Applied Maths Learning at Scale are set to play a transformative role in the “internet of things”. This is a practical module that looks at how ML is applied to internet-scale systems. Topics covered will include A/B testing, ranking, recommender systems, and the modelling of users and entities that they engage with online (like news stories). Network effects, social networks, online advertising, and ML for real-time auctions will also be covered. Computer Vision Computer Vision has long been an important driving force for advances in Machine 15 Applied Maths Learning and have been instrumental in the rise and development of deep learning. The module will start with convolutional neural networks for image classification, and extensions like dropout, batch normalisation, data augmentation, transfer learning, and visual attention. Other typical Computer Vision tasks will then be overviewed, including object segmentation, colourisation, style transfer, and automated image captioning. Finally generative models for Computer Vision, including variational autoencoders and generative adversarial networks, will be covered. Natural Language Parsing, understanding, and generating natural human language is a crucial 15 Applied Maths Processing component of AI. Advances in deep learning are beginning to enable impressive end-to-end language understanding systems. Topics in this module will include word embeddings and representations (e.g. word2vec), part-of-speech tagging and syntactic parsing, topic modelling, language modelling with recurrent and convolutional neural networks, machine translation with seq2seq models and attention, and sentence classification in applications like sentiment labelling and language identification. Sequence Modelling This module deals with techniques to model and predict temporally varying data, 15 Applied Maths such as financial time series data, weather data, audio and video signals. It begins with classical state-space models, linear as well as hidden Markov models, then recurrent neural networks and the most common modular ways of constructing them, e.g. with long short-term memory gates. Concurrent neural network architectures, like the Transformer model, that processes the symbols of an entire sequence concurrently (or in a series of concurrent passes) will also be covered, and the module ends with methods to combine the above.

5 Module Description Credits Responsible Monte Carlo Monte Carlo methods form a family of stochastic approximation techniques, and 15 Applied Maths Methods are widely used in fields spanning Physics, Statistics and Finance. It is also a principal tool for statistical inference in ML. The module will cover Markov chains and Monte Carlo methods. These include Metropolis-Hastings methods, Gibbs sampling and collapsed Gibbs sampling, with applications to topic modelling and non-parametric Bayesian inference. Topics like importance sampling, slice sampling and exact sampling will also be included. Inference techniques for normalising constants (model scores), annealing and thermodynamic integration will also be covered. Project Data Science Students will be required to apply and consolidate the knowledge gained throughout 60 Host Industrial Project this program. For this purpose, students will solve a real-world engineering data Engineering, science project, providing solutions for each step of the data science project life with study cycle. As outcome of this project, students will produce a dissertation, describing leaders in all all of the life cycle phases and research conducted in order to provide a solution to engineering the specific data science problem. departments It is encouraged that the knowledge gained in specialisation module selected above, and Computer is utilised in the project. Science

he rapidly progressed through the ranks, his last two Academic Profiles of Core Module positions being Head of Department (2009 to 2017) and Presenters Director: Institute for Big Data and Data Science (2017 to 2018). After 21 years at Tuks, the opportunity arose to Prof Andries Engelbrecht return to his alma mater. His appointment at Stellenbosch is an A-rated researcher as University comprises two aspects; 50% is allocated to his rated by die National Research role as Chair in Data Science in the Faculty of Engineering Foundation (NRF). This rating and 50% as an academic in the Department of Computer acknowledges that he is a Science in the Faculty of Science. leading international researcher "Regarding my position as Chair, my main aim is to promote in his field. His fields of Data Science within Stellenbosch University. This includes the expertise are Computational transfer of knowledge to undergraduate and postgraduate intelligence, Swarm intelligence, students as well as to industry. In order to do this, I want to Evolutionary computation, Neural networks, establish a research group within the Department of Industrial Optimisation, Machine learning and Data analytics. Engineering. I have two bright young colleagues in my team, Prof Engelbrecht is the first incumbent of the new Voigt Prof Jacomine Grobler and Dr Thorsten Schmidt-Dumont. We Chair in Data Science in the Department of Industrial already have quite a number of master's students enrolled for Engineering. 2019.” As a first-year Matie student in 1988 he opted for BSc He presents Data analytics to third-year industrial with Computer Science and Mathematics. He obtained his engineering students and plans to lead this program as the honours cum laude in 1992, followed by a master's cum academic program head. laude (1994) and a PhD in 1999. The research interests of Prof Jan H During his postgraduate studies he taught Computer van Vuuren are combinatorial Studies and Mathematics on a temporary basis at a few optimisation and decision support schools. His introduction to academia came with his within the wider area of operations appointment as a lecturer in Computer Science at Unisa research. He heads the Stellenbosch in 1996 where he stayed for two years. Unit for Operations research in Engineering (SUnORE) within the His academic career gained momentum in 1998 when he Department of Industrial Engineering. joined the University of Pretoria as a lecturer in Computer Science. Over the two decades that followed 6 He obtained his bachelor's degree in science cum laude Dr Thorsten Schmidt-Dumont from Stellenbosch University in 1989, majoring in is the first recipient of a postdoctoral mathematics and applied mathematics. He followed this fellowship with a focus on machine up with an honours degree cum laude in 1990 and a learning applications within the master's degree cum laude in 1992, both in applied Department of Industrial Engineering. mathematics and both from Stellenbosch University. He His areas of expertise include the obtained his doctorate in mathematics from the application of reinforcement learning University of Oxford, United Kingdom in 1995. He has algorithms to complex problems. been a member of staff at Stellenbosch University since Additionally, he has a background in 1996, first in the Department of Applied Mathematics approximate optimisation techniques, particularly (until 2007) and then at the Department of Logistics (until metaheuristics, and computer simulation. 2013). He is currently professor of operations research Dr Schmidt-Dumont was born and raised in Namibia, within the Department of industrial Engineering, a before moving to Stellenbosch to start his tertiary position he has held since 2014. In all of these education, opting for a degree in Industrial Engineering departments he has taught subjects related to based on the wide applicability of the taught skillset in optimisation, at both undergraduate and postgraduate various industry sectors. He obtained his degrees from levels. Stellenbosch University (SU): BEng cum laude 2015, PhD His academic passions are research and postgraduate 2018. students. He is the author of close to a hundred journal Highlights throughout his under- and postgraduate studies publications and has supervised (or co-supervised) 29 include being awarded the prize for the best final-year doctoral students to the successful completion of their project in Industrial Engineering in 2015, a project for studies. which he was also awarded the prestigious Gerhard Prof Jacomine Grobler joined Geldenhuys Medal by the Operations Research Society of the Department of Industrial South Africa for the best 4th year project countrywide in Engineering at Stellenbosch on 1 Operations Research completed in 2015. Buoyed by this February 2019 as Associate success at undergraduate level he started his postgraduate Professor. Her areas of expertise journey, completing a master’s degree, which was are the development of designated as the runner-up for the prestigious Theodor optimisation algorithms and the Stewart medal, awarded by the Operations Research application of data science within Society of South Africa, for the best master’s project the field of industrial engineering. With this background countrywide in 2018, which he was able to upgrade to a she joined the newly-established Data Science research PhD in 2018. group. During the final year of completing his PhD he was Prof Grobler was born and raised in Pretoria. She decided appointed as a part-time junior lecturer in Industrial to study engineering as the work is creative and requires Engineering at Stellenbosch University, teaching industrial good problem-solving skills. Furthermore, she opted for programming at second year level. Industrial Engineering in particular, because it is people oriented and there are vast opportunities in industry for Admission Requirements and Fees industrial engineers. She obtained all her industrial engineering degrees at the To be considered for admission you must: University of Pretoria (UP): BEng 2006, HonsBEng 2007, • Hold at least a BEng, a BScHons, another relevant MEng 2009 and PhD 2015. four-year bachelor’s degree, an MTech, or a PGDip During her study career, she received 12 awards. These (Eng); or include the Department of Industrial and System • Hold other academic degree qualifications and Engineering prize for the best final-year project; Medal appropriate experience that have been approved by from the South African Institute of Industrial Engineering the Faculty Board. The department’s chairperson for the best final-year Industrial Engineering student; must make a recommendation regarding such a Winner of the SAS Operations Research National Student qualification and experience to the Faculty Board. Competition best honours project; South African Students must have passed 1st year Mathematics, Statistics Association for the Advancement of Science Bronze or Applied Mathematics. Computer programming medal for the best dissertation at master’s level at the experience is also an advantage. University of Pretoria; and the 2017 South African Also refer to the post graduate admission model in Figure Institute for Industrial Engineering Award for Outstanding 3.1, in Section 3.2 in the Engineering Calendar. Young Industrial Engineering Researcher. Fees are adjusted annually, and the expected fees for 2020 From August 2008 to March 2014 she was employed by Denel Dynamics Pty Ltd. Thereafter she joined the CSIR will thus be adjusted in line with university policy. The as research group leader: Transport and Freight Logistics. current fees for an MEng (Structured) in 2019 are shown In October 2015 she returned to academia, this time as in the table below. lecturer in supply chain management at UP. This means that if you study full-time and manage to complete the program in 1 year, your total fee will be

7 R27 008 + R55 800 (for 180 credits at R310 per credit) • Year 2: R27 008 + R18 600 = R45 608 for a total fee of R82 808. • Year 3: R27 008 + R18 600 = R45 608 A more reasonable approach may be to spread this over • Total fee: R136 824 2 years, but then there will be an additional registration The main difference between part-time and full-time is the fee of R27 008. The fee will then be: length of time you are allowed to enroll. We expect a full- • Year 1: R27 008 + R27 900 = R54 908 time student to finish the program within 2 years, but • Year 2: R27 008 + R27 900 = R54 908 he/she may apply for a 3rd year, with a good motivation. A • Total fee: R109 816 part-time student is allowed to be registered for a longer period, as shown in the above table. Similarly, if the program is spread over 3 years part-time:

• Year 1: R27 008 + R18 600 = R45 608

Registration Year 1 2 3 4 5 6 Full-time enrolment R27 008 R27 008 R29 709 NA NA NA Plus Cost per credit R310 R310 R310 NA NA NA Part-time enrolment R27 008 R27 008 R27 008 R29 709 R32 679 R35 947 Plus cost per credit R310 R310 R310 R310 R310 R310

you with filling in the application forms and will submit your application for the acceptance process. Program Status

This program is in a “pending final approval” status (May Contacts 2019). Final approval will only happen in November 2019, • Data Sciences Program Head: but we believe this is a low risk. Prof Andries Engelbrecht: [email protected] The first intake for January 2020 will be limited to 30 • Postgraduate Manager: students. Melinda Rust: [email protected] • Other contacts: Enrolment • Prof Jan van Vuuren: [email protected] • Prof Jacomine Grobler: [email protected] For general enquiries about the program, and to find out • Dr Thorsten Schmidt-Dumont: [email protected] more, contact Prof Andries Engelbrecht. • Departmental Chair: To assist you with the enrolment process, contact Prof Corne Schutte: [email protected] Melinda Rust, our Postgraduate Manager. She will assist • Tel nr: (021) 808 4234 • Web: www.ie.sun.ac.za

Version 4 June 2019 8