Linear Algebra for Chemical Engineers

LINEAR ALGEBRA FOR CHEMICAL ENGINEERS

KYRIACOS ZYGOURAKIS TABLE 1 Rice University Course Materials Houston, TX 77251 COURSE TEXTBOOK

FIRST-YEAR GRADUATE course (or sequence of Strang, G., Linear Algebra and Its Applications, 2nd A courses) in applied mathematics has become Edition, Academic Press, (1980). an integral part of the curriculum in a large ADDITIONAL COURSE REFERENCES number of chemical engineering departments. Among the diverse subjects taught in these 1. A mundson, N. R., Mathematical Methods in Chemical Engineering: Matrices and Their Application, Pren courses, linear algebra usually enjoys a prominent tice Hall, (1966). position. The reason for this popularity perhaps 2. Braun, M., Differential Equations and Their Applica lies in the fact that linear algebra is as central a tions, 2nd Edition, Springer-Verlag (1975). subject and as applicable as calculus. The pioneer 3. Dahlquist, G., A. Bjorck and N. Anderson, Numerical ing work of Neal Amundson, and of his students Methods, Prentice Hall (1974). and disciples as well as other prominent scholars, 4. Friedman, B., Principles and Techniques of Applied Mathematics, John Wiley (1956). has established beyond any doubt that many sig 5. Hirsch, M. W. and S. Smale, Differential Equations, nificant and complex chemical engineering prob Dynamical Systems and Linear Algebra, Academic lems may be solved by advanced linear algebra Press (1974). techniques [1]. 6. Noble, B. and J. W. Daniel, Applied Linear Algebra, Linear algebra can also serve as an ideal 2nd Edition, Prentice Hall (1977). 7. Steinberg, D. T., Computational Matrix Algebra, Mc stepping stone for introducing the first-year Graw-Hill (1974). graduate student to the formal mathematical language of functional analysis. The basic con The theory, however, is motivated and reinforced cepts of matrix algebra, already familiar to the by examples derived from a wide range of chemi student, can be formulated using the abstract cal engineering problems. Particular emphasis is framework of linear vector spaces. The same placed upon the important aspects of computa abstraction can also be used to unify apparently tional linear algebra. In our opinion, it is impera diverse problems in finite dimensional spaces tive to expose the students to some fundamental under this common framework. Thus, the ground computational methods and to study their efficiency work is laid out for the introduction of functional as well as their convergence problems. Student analysis in infinite dimensional spaces, which is responses to the course evaluation questionnaire necessary for the study of differential and integral indicate that they particularly enjoy the compu operator problems [2]. tational part of the course since it points out some Our linear algebra course strives to combine of the real problems to which linear algebra theory both elements of mathematics-abstraction and can be applied. application. Many of the fundamental theorems of linear algebra are rigorously derived in class. COURSE ORGANIZATION Eleven weeks ( out of a total of fifteen) of Student responses to the course the fall semester course, "Applied Mathematics evaluation questionnaire indicate that for Chemical Engineers I," are devoted to the they particularly enjoy the computational part of the course since it points out some of the study of linear algebra and its applications. The real problems to which linear algebra remaining time is devoted to a brief review of theory can be applied. complex analysis and complex integration, which is the final preparation step for the second course

176 CHEMICAL ENGINEERING EDUCATION signed to help the students with their computer projects as well as for the discussion of home work assignments in an informal way. The students are urged to keep a complete set of notes, which are regularly supplemented by handouts providing lengthy theorem proofs or summarizing the results established up to that point. The assigned textbook is Linear Algebra and its Applications (2nd Edition), by Gilbert Strang. Although it is an extremely well-written book, it is not followed closely ( especially in the first part of the course). The students are strongly en Kyriacos Zygourakis received his diploma in chemical engineering couraged to consult additional references (see from the National Technical University of Greece in 1975 and his PhD Table 1) . from the University of Minnesota in 1980. He is presently an assistant Homework problems are assigned almost professor in the Department of Chemical Engineering at Rice Uni every week. In addition, the students are required versity. His main research interests are in the areas of reaction engineering, applied mathematics and numerical methods. to complete one or two computational projects. They also have to take a mid-semester and a final exam, which consist of both open- and closed-book second course covers the theory of differential and parts. integral operators, again using the functional analysis approach. COURSE CONTENTS The course meets twice a week for two hours and runs largely as a lecture, although active The linear algebra part of the course (see student participation is encouraged by frequent Table 2) consists of four parts: questions from the instructor. The lectures are • Vector spaces and linear transformations accompanied by tutoring sessions which are de- • The solution of systems of linear equations

TABLE 2 Topical Outline of the Linear Algebra Course

1. VECTOR SPACES AND LINEAR • Overview of iterative methods for solving linear TRANSFORMATIONS equations. • Comparison of the various numerical algorithms. • Overview of the problem of solving systems of linear equations. Which applications give rise 3. THE EIGENVALUE PROBLEM A x = AX to such systems? Which are the theoretical porblems that must be answered? • Determinants. • Vector spaces and subspaces. • Inner products, norms, orthogonality. • Linear dependence, basis and dimension. • Eigenvalues and eigenvectors of matrices. • Linear transformations between finite-dimension • Diagonalization and similarity transformations. al spaces and their matrix representation. • Systems of difference equations. • Rank and nullity of linear transformations. • Functions of matrices. • Elementary matrices and the computation of the • Solution of systems of ordinary differential rank of a matrix. equations. Stability. • The theory of simultaneous linear equations. • Unitary transformations. Normal matrices. Homogeneous and nonhomogeneous systems. • Spectral decomposition of operators. The Fredholm alternative. 4. QUADRATIC FORMS AND VARIATIONAL 2. SOLUTION OF SYSTEMS OF LINEAR PRINCIPLES EQUATIONS A x = b • Positive definite quadratic forms. • Gaussian elimination. LU---decomposition, pivot • Minimization problems. Least squares. ing, operation count. • Rayleigh quotient. Maximum and minimax • Error analysis. Ill-conditioned matrices. principles. • Band matrices and how they arise in practice. • Numerical computation of eigenvalues and Finite differences solution of partial differ eigenvectors. ential equations. • Overview of the finite elements method.

FALL 1984 177 The students are thus presented with our objectives for the first part of the course. A brief review of the algebra of matrices follows, reminding the student of the familiar concepts of multiplying a matrix by a scalar to obtain another matrix and of summing two matrices to obtain a third one.

• The Eigenvalue problem problem of linear algebra A x = b. This is ac • Quadratic forms and variational principles complished in one lecture using the previously The Linear Equation Problem A x = b derived theorems. Throughout this part of the course, emphasis The course starts with an introduction to the is placed on the generality of this approach, and problem of solving systems of linear equations of the students have the opportunity to see how the the form A x = b. Several applications that give results apply to linear differential and integral rise to such large systems are discussed and the operators, as well as to chemical engineering three fundamental questions are introduced: problems. Such examples include first-order re • Do these problems have a solution? action systems and the determination of the • If they do, is the solution unique? number of independent chemical reactions in a • How can the solution be computed? closed system using experimental measurements. The students are thus presented with our ob The practical problem of efficiently computing jectives for the first part of the course. A brief the solution of systems of linear equations can now review of the algebra of matrices follows, remind be considered. The Gauss elimination procedure ing the student of the familiar concepts of multi and the LU decomposition are introduced, which plying a matrix by a scalar to obtain another lead naturally to the idea of the operation count matrix and of summing two matrices to obtain a as a measure of the computational effort required. third one. It is also pointed out that these opera An important application which gives rise to large tions satisfy certain properties such as associativi systems of linear equations is then studied by ty, commutativity, distributivity, etc. This dis introducing the finite-difference method for solv cussion serves as the motivation to introduce the ing ordinary and partial differential equations notion of abstract linear vector spaces. Several subject to specified boundary conditions. The examples of vector spaces are then presented, students learn how to take advantage of the covering sets of functions, polynomials, solutions matrix structure (band or positive-definite of differential or integral equations, etc. The matrices) in order to speed up the computational students come to realize that seemingly different process and how to use the LU-decomposition for mathematical systems may be considered as the efficient solution of iterative problems that vector spaces and that this abstract framework arise in the solution of nonlinear differential equa can unify these diverse phenomena into a single tions. The problem of ill-conditioned matrices is study. outlined in sketchy form, along with a rudimentary The basic concepts of linear combinations, basis introduction to error analysis. Iterative methods sets, and dimension are then discussed. Thus, the for the solution of linear systems of equations are abstract quantities called vectors can be repre also briefly covered. sented now in terms of their coefficients of ex At this point a computer project is assigned. pansion with respect to a particular basis set. The students are asked to solve a two-dimensional The first milestone is reached with the intro partial differential equation using finite differ duction of linear transformations between finite ences. They must use different grid sizes and dimensional spaces and their matrix representa compare the numerical results to the true solutions tion. Most of the important theorems here are in each case. rigorously derived in class and the concepts of The students must demonstrate that they can rank and nullity of transformations are formally correctly formulate the system of linear equations. introduced. Armed with the conclusion that all Following that, they use the library programs the results established for linear transformations available at our computer center to obtain the can be used for matrices (and conversely), we can results. The library programs LINPACK and then establish the conditions for existence and ITPACK (for the direct and iterative solution of uniqueness of solutions of the first fundamental linear systems) have proven to be invaluable aids.

178 CHEMICAL ENGINEERING EDUCATION Thus, the emphasis is shifted from the drudg differential equations. The cases of operators with ery of computer programming to the analysis of distinct and non-distinct eigenvalues are treated the results. The numerical simulations permit the in detail, although the case of defective matrices students to evaluate the relative efficiency of and the Jordan canonical form are only briefly numerical schemes (i.e. execution speeds, memory covered. requirements) and to determine which ones must Throughout this part of the course it is con be used for the various structures and sizes of the tinuously emphasized that the eigenvalues are resulting matrices. Thus, the theoretical results the most important feature of any dynamical derived in class are reinforced and justified. system. The students have the opportunity to The second part of the computer assignment solve a large variety of chemical engineering exposes the students to the pitfalls which may be problems. They study: fall the unwary and uninstructed user of computer • The difference equations describing a cascade of software packages. The students are asked to CSTR'S. solve a system of equations for which the matrix • The differential equations describing isothermal and of the coefficients of the unknowns is badly ill nonisothermal CSTR's and their stability. conditioned (the notorious Hilbert matrix has • The problem of N first-order chemical reactions served as the perfect example in this respect). The taking place in a catalyst pellet. • The difference equations resulting when a con students are asked to compute the known solution tinuous system is subject to piecewise constant inputs, of a system of equations using single and double which provides them with an introduction to sampled precision computer arithmetic. They are then data system theory. asked to explain why the solution deteriorates as • The problem of N first-order reactions taking place the order of the system increases by monitoring in a batch reactor. This is a long assignment, which the magnitude of the pivoting elements, the con dition number of the matrix, and using the theory Throughout this part of the presented in class. course it is continuously emphasized that the eigenvalues are the most important feature of any dynamical system. The Eigenvalue Problem A x = Xx The second part of the course starts with a leads the students in a step-by-step fashion to derive brief review of the theory of determinants. Their the theoretical results necessary to determine all the properties are presented along with the basic rate constants, through a set of carefully designed formulas for their computation. The operation experiments [3]. This problem encompasses almost count for solving systems of linear equations using everything the students have learned so far in the Cramer's rate is derived and most of the students course. As such, it has come to be known as the are surprised to find out that even the most power "Everything you always wanted to know about first order reactions in batch ( ... and more!)" assign 1 45 ful computer would need about 10 years to solve ment. a 100 x 100 system using this method. They are reminded, however, that determinants give a very The final part of the course introduces the useful invertibility test for square matrices, whose students to the concept of formulating the two main application will be used later on in the main problems of linear algebra, namely A x = b course for the development of the theory of eigen and A x = Xx, as minimization problems. The values. The concepts of inner products of vectors emphasis now shifts to pointing out the ad and of the norm of a vector are then presented vantages of this approach for numerical computa as abstract mappings of vectors into the field of tions. The problem of minimization of a multi real (or complex) numbers and are related to the variable function serves as the starting point for familiar notions of angle between vectors and of an introduction of the concepts of quadratic forms magnitude respectively. and positive definite matrices. The least squares A discussion of the solution of a simple 2 x 2 method is then developed formally, and its practi system of linear ordinary differential equations cal implications are considered. The course closes motivates the introduction of the eigenvalues of a with the formulation of the eigenvalue problem matrix A. The main emphasis here is on the de as a minimization one. The Rayleigh and the mini velopment of the theoretical results needed for max principles are presented, followed by a brief the solution of systems of difference and ordinary Continued on page 213.

FALL 1984 179 TRANSPORT PHENOMENA LINEAR ALGEBRA Cont:nued from page 173. Continued from page 179. this method. However, emphasis is placed on when discussion of simple numerical methods for the such an approximation can be invoked by develop computation of eigenvalues. In order to further ing ideas on multiple time scale analysis. The establish the importance of the variational method is illustrated by considering shrinking methods, the finite element method is briefly out unreacted core model in gas-solid reactions and lined at the end of the course, using tools that evaporation of a drop in a stagnant fluid. the students already possess. Additional topics covered in the course are listed in Table 3. These include non-Newtonian CONCLUDING REMARKS fluid flow, turbulent flow, some cases of exact solution of Navier-Stokes equations, evaluations Our course attempts to introduce the students of Nusselt and Sherwood numbers in laminar and to the essentials of linear algebra and, at the turbulent flow, and some cases of mass transfer same time, to convey the fact that these elegant where no analogs in heat transfer are available. results can be applied to a wide range of engineer Finally, some examples of macroscopic balances ing problems. Significant emphasis is placed upon are also solved. the development of basic and efficient compu tational methods. There is hardly any need to SUMMARY stress again the importance of exposing the chemi The course is essentially a survey in transport cal engineering graduate student to the basics of processes. An attempt is made to give students a numerical analysis. Our experience indicates that thorough understanding of the topics covered, so the essentials of computational linear algebra can that they can formulate the necessary differential be successfully integrated into an applied mathe equations. They are given sufficient insight into matics course. A large number of students go on some of the powerful tools available to analyze to take a rigorous numerical analysis course given and solve these equations. It is emphasized that by the Mathematical Sciences Department at Rice, the nnswers obtained must be checked to see if which covers methods for the solution of ordinary the assumptions made in deriving them are ful and partial c. ifferential equations. They have dis filled. It is also stressed that in most cases, knowing covered that their background in computational the distribution of velocity, temperature, and con linear algebra was adequate. centration is not as important as knowing the We plan to introduce still another computer fluxes at the interface. These in turn are then\ project in future offerings of this course, in order related to friction factor, Nusselt, and Sherwood to familiarize the students with some of the most numbers respectively. The course as described useful methods for the numerical computation here has been well received by the students. Good of eigenvalues and eigenvectors of large matrices. students tend to feel they are ready to tackle more The emphasis will again be on the understanding difficult topics. Terminal master's students feel of the physical problem and the resulting mathe they have a solid foundation in transport phe matical one, and on the study of the relative ad nomena on which they can continue to build their vantages of the various algorithms. • practical experience. • REFERENCES ACKNOWLEDGMENT 1. Bird, R. B., W. E. Stewart, E. N. Lightfoot, Transport The author wishes to acknowledge the in Phenomena, 7th printing, Wiley, New York, 1960. fluence of his mentors, Rutherford Aris, Neal 2. Bird, R. B., W. E . Stewart, E. N. Lightfoot, and Amundson and D. Ramkrishna, who have shown T. W. Chapman, AIChE Continuing Education Series, No. 4, 1969. him that applied mathematics can also be enjoy 3. "Selected Topics in Transport Phenomena," Chem. able and who have shaped his ideas about teach Eng. Symp. Ser., No. 58, 61, 1965. ing. 4. Denn, M. M., Process Fluid Mechanics, Prentice-Hall, Inc., Englewood Cliffs, N.J., 1980. REFERENCES 5. Schlichting, H., Boundary-Layer Theory, 7th Edition, McGraw-Hill, New York, N.Y., 1979. 1. Amundson, N. R., Chem. Eng. Edn., 3, 174 (1969). 6. Slattery, J.C., Momentum, Energy and Mass Transfer 2. Ramkrishna, D., Chem. Eng. Edn., 13, 172 (1979). in Continua, Robert E. Kreiger Publishing Company, 3. Wei, T. and C. D. Prater, Adv. Catalysis, 13, 204 2nd Edition, Huntington, N.Y., 1981. (1962).

FALL 1984 213