DOCSLIB.ORG

Giao Trinh Giai Tich Da

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Giao Trinh Giai Tich Da BỘ SÁCH TOÁN CAO CẤP - VIỆN TOÁN HỌC NGUYỄN ĐÔNG YÊN GIÁO TRÌNH GIẢI TÍCH ĐA TRỊ nhà xuất bản khoa học tự nhiên và công nghệ SÁCH Đà IN TRONG BỘ NÀY: 2000: Phương trình vi phân ₫ạo hàm riêng (Tập 1) Trần Đức Vân 2001: Giáo trình Đại số tuyến tính Ngô Việt Trung Phương trình vi phân ₫ạo hàm riêng (Tập 2) Trần Đức Vân Nhập môn Lý thuyết ₫iều khiển Vũ Ngọc Phát 2002: Giải tích các hàm nhiều biến Đ.T. Lục, P.H. Điển,T.D. Phượng Lý thuyết Hệ ₫ộng lực Nguyễn Đình Công 2003: Lôgic toán và Cơ sở toán học Phan Đình Diệu Giáo trình Đại số hiện ₫ại Nguyễn Tự Cường Lý thuyết không gian Orlicz Hà Huy Bảng Đại số máy tính: Cơ sở Groebner Lê Tuấn Hoa Hàm thực và Giải tích hàm Hoàng Tụy Số học thuật toán H.H. Khoái, P.H. Điển 2004: Mã hóa thông tin: Cơ sở toán học và ứng dụng P.H. Điển, H.H. Khoái Lý thuyết Tổ hợp và Đồ thị Ngô Đắc Tân Xác suất và Thống kê Trần Mạnh Tuấn 2005: Giải tích Toán học: Hàm số một biến Đ.T. Lục, P.H. Điển, T.D. Phượng Lý thuyết Phương trình vi phân ₫ạo hàm riêng (Toàn tập) Trần Đức Vân Công thức kiểu Hopf-Lax-Oleinik cho phương trình Hamilton-Jacobi Trần Đức Vân Đại số tuyến tính qua các ví dụ và bài tập Lê Tuấn Hoa Lý thuyết Galois Ngô Việt Trung 2007: Lý thuyết tối ưu không trơn N.X. Tấn, N.B. Minh Giáo trình Giải tích ₫a trị Nguyễn Đông Yên Có thể đặt mua sách trực tiếp tại Viện Toán học, 18 Hoàng Quốc Việt, Hà Nội Điện thoại 84-4-7563474/205 (Văn phòng); 84-4-7563474/302 (Thư viện) Fax: 84-4-7564303 E-mail: [email protected] (VP), [email protected] (TV) Lời giới thiệu rong những năm gần đây, nhu cầu sách tham khảo tiếng Việt về toán của sinh viên các trường Ðại học, nghiên cứu sinh, cán bộ nghiên cứu Tvà ứng dụng toán học tăng lên rõ rệt. Bộ sách "Toán cao cấp" của Viện Toán h ọc ra đời nhằm góp phần đáp ứng yêu cầu đó, làm phong phú thêm nguồn sách tham khảo và giáo trình đại học vốn có. Bộ sách Toán cao cấp sẽ bao gồm nhiều tập, đề cập đến hầu hết các lĩnh vực khác nhau của toán học cao cấp, đặc biệt là các lĩnh vực liên quan đến các hướng đang phát triển mạnh của toán học hiện đại, có tầm quan trọng trong sự phát triển lý thuyết và ứng dụng thực tiễn. Các tác giả của bộ sách này là những người có nhiều kinh nghiệm trong công tác giảng dạy đại học và sau đại học, đồng thời là những nhà toán học đang tích cực nghiên cứu. Vì thế, mục tiêu của các cuốn sách trong bộ sách này là, ngoài việc cung cấp cho người đọc những kiến thức cơ bản nhất, còn cố gắng hướng họ vào các vấn đề thời sự liên quan đến lĩnh vực mà cuốn sách đề cập đến. Bộ sách Toán cao cấp có được là nhờ sự ủng hộ quý báu của Viện Khoa học và Công nghệ Việt Nam, đặc biệt là sự cổ vũ của Giáo sư Ðặng Vũ Minh và Giáo sư Nguyễn Khoa Sơn. Trong việc xuất bản Bộ sách, chúng tôi cũng nhận được sự giúp đỡ tận tình của Nhà xuất bản Ðại học quốc gia Hà Nội và của Nhà xuất bản Khoa học Tự nhiên và Công nghệ. Nhiều nhà toán học trong và ngoài Viện Toán học đã tham gia viết, thẩm định, góp ý cho bộ sách. Viện Toán học xin chân thành cám ơn các cơ quan và cá nhân kể trên. Do nhiều nguyên nhân khác nhau, Bộ sách Toán cao cấp chắc chắn còn rất nhiều thiếu sót. Chúng tôi mong nhận được ý kiến đóng góp của độc giả để bộ sách được hoàn thiện hơn. Chủ tịch Hội ₫ồng biên tập GS-TSKH Hà Huy Khoái BỘ SÁCH TOÁN CAO CẤP - VIỆN TOÁN HỌC HỘI ĐỒNG BIÊN TẬP Hà Huy Khoái (Chủ tịch) Ngô Việt Trung Phạm Huy Ðiển (Thư ký) GIÁO TRÌNH GIẢI TÍCH ĐA TRỊ Nguyễn Đông Yên Viện Toán học, Viện KH&CN Việt Nam NHÀ XUẤT BẢN KHOA HỌC TỰ NHIÊN VÀ CÔNG NGHỆ Môc lôc Lêi nãi ®Çu 3 C¸c ký hiÖu vµ ch÷ viÕt t¾t 6 1 TÝnh liªn tôc cña ¸nh x¹ ®a trÞ 9 1.1 ¸nhx¹®atrÞ............................ 9 1.2 TÝnh nöa liªn tôc trªn vµ tÝnh nöa liªn tôc d−íi cña ¸nh x¹ ®a trÞ 18 1.3 §Þnh lý Kakutani . ......................... 27 1.4 C¸c qu¸ tr×nh låi . ......................... 37 1.5 C¸c tÝnh chÊt Lipschitz cña ¸nh x¹ ®a trÞ ............. 45 2 §¹o hµm cña ¸nh x¹ ®a trÞ 47 2.1 Nguyªn lý biÕn ph©n Ekeland ................... 47 2.2 Nãn tiÕp tuyÕn . ......................... 53 2.3 §¹ohµm.............................. 71 3 TÝch ph©n cña ¸nh x¹ ®a trÞ 77 3.1 ¸nh x¹ ®a trÞ ®o ®−îc, l¸t c¾t ®o ®−îc.............. 77 3.2 TÝch ph©n cña ¸nh x¹ ®a trÞ .................... 91 3.3 L¸t c¾t liªn tôc vµ l¸t c¾t Lipschitz . .............. 95 3.4 TÝch ph©n Aumann cña ¸nh x¹ d−íi vi ph©n Clarke ....... 98 4 §èi ®¹o hµm cña ¸nh x¹ ®a trÞ 103 4.1 Sù ph¸t triÓn cña lý thuyÕt ®èi ®¹o hµm ..............104 4.2 C¸c kh¸i niÖm c¬ b¶n cña lý thuyÕt ®èi ®¹o hµm .........106 4.3 VÊn ®Ò ®¸nh gi¸ d−íi vi ph©n cña hµm gi¸ trÞ tèi −u.......116 4.4 TÝnh comp¾c ph¸p tuyÕn theo d·y . ..............118 4.5 D−íi vi ph©n FrÐchet cña hµm gi¸ trÞ tèi −u ...........120 4.6 D−íi vi ph©n Mordukhovich cña hµm gi¸ trÞ tèi −u........136 4.7 D−íi vi ph©n Mordukhovich cña phiÕm hµm tÝch ph©n . 148 1 2 5 HÖ bÊt ®¼ng thøc suy réng 153 5.1 Giíi thiÖu chung . .........................154 5.2 C¸c ®Þnh nghÜa vµ kÕt qu¶ bæ trî . ..............155 5.3 TÝnh æn ®Þnh . .........................160 5.4 Quy t¾c nh©n tö Lagrange . ....................174 5.5 TÝnh liªn tôc vµ tÝnh Lipschitz cña hµm gi¸ trÞ tèi −u.......178 5.6 Chøng minh MÖnh ®Ò 5.2.1 ....................183 5.7 D−íi vi ph©n Mordukhovich vµ d−íi vi ph©n J-L .........186 5.8 §èi ®¹o hµm Mordukhovich vµ Jacobian xÊp xØ .........194 Phô lôc A 201 Phô lôc B 203 Tµi liÖu tham kh¶o 205 Danh môc tõ khãa 215 3 Lêi nãi ®Çu Gi¶i tÝch ®a trÞ lµ mét h−íng nghiªn cøu t−¬ng ®èi míi trong To¸n häc, mÆc dï tõ nh÷ng n¨m 30 cña thÕ kû XX c¸c nhµ to¸n häc ®· thÊy cÇn ph¶i nghiªn cøu ¸nh x¹ ®a trÞ, tøc lµ ¸nh x¹ nhËn gi¸ trÞ lµ c¸c tËp hîp con cña mét tËp hîp nµo ®ã. Sù ra ®êi cña t¹p chÝ quèc tÕ “Set-Valued Analysis” vµo n¨m 1993 lµ mét mèc lín trong qu¸ tr×nh ph¸t triÓn cña h−íng nghiªn cøu nµy. Vai trß cña gi¶i tÝch ®a trÞ trong To¸n häc vµ c¸c øng dông to¸n häc ®· ®−îc c«ng nhËn réng r·i. Gi¶i tÝch ®a trÞ cã nhiÒu øng dông trong lý thuyÕt ph−¬ng tr×nh vi ph©n, ph−¬ng tr×nh ®¹o hµm riªng, bÊt ®¼ng thøc biÕn ph©n vµ ph−¬ng tr×nh suy réng, lý thuyÕt tèi −u, lý thuyÕt ®iÒu khiÓn, tèi −u ®a môc tiªu, khoa häc qu¶n lý, vµ to¸n kinh tÕ. HiÖn nay hÇu nh− tÊt c¶ c¸c kÕt qu¶ nghiªn cøu vÒ tÝnh æn ®Þnh vµ ®é nh¹y nghiÖm cña c¸c bµi to¸n tèi −u phô thuéc tham sè vµ cña c¸c bµi to¸n bÊt ®¼ng thøc biÕn ph©n phô thuéc tham sè ®Òu ®−îc viÕt b»ng ng«n ng÷ gi¶i tÝch ®a trÞ. Nh÷ng ng−êi ViÖt Nam ®Çu tiªn ®i s©u nghiªn cøu gi¶i tÝch ®a trÞ lµ Gi¸o s− Hoµng Tôy (víi nh÷ng c«ng tr×nh vÒ ®iÓm bÊt ®éng cña ¸nh x¹ ®a trÞ, tÝnh æn ®Þnh cña hÖ bÊt ®¼ng thøc suy réng, ¸nh x¹ ®a trÞ låi, ¸nh x¹ tíi h¹n), Gi¸o s− Ph¹m H÷u S¸ch (víi nh÷ng c«ng tr×nh vÒ ¸nh x¹ ®a trÞ låi, ®¹o hµm cña ¸nh x¹ ®a trÞ vµ øng dông trong lý thuyÕt tèi −u vµ ®iÒu khiÓn) vµ cè Gi¸o s− Phan V¨n Ch−¬ng (víi nh÷ng c«ng tr×nh vÒ ¸nh x¹ ®a trÞ ®o ®−îc, lý thuyÕt bao hµm thøc vi ph©n). Sau ®©y lµ danh s¸ch kh«ng ®Çy ®ñ nh÷ng ng−êi ViÖt Nam ®· hoÆc ®ang cã c«ng tr×nh nghiªn cøu vÒ gi¶i tÝch ®a trÞ vµ c¸c øng dông: Th.S. Ph¹m Ngäc Anh, Th.S.
Load more

Recommended publications
  • CORE View Metadata, Citation and Similar Papers at Core.Ac.Uk
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Bulgarian Digital Mathematics Library at IMI-BAS Serdica Math. J. 27 (2001), 203-218 FIRST ORDER CHARACTERIZATIONS OF PSEUDOCONVEX FUNCTIONS Vsevolod Ivanov Ivanov Communicated by A. L. Dontchev Abstract. First order characterizations of pseudoconvex functions are investigated in terms of generalized directional derivatives. A connection with the invexity is analysed. Well-known first order characterizations of the solution sets of pseudolinear programs are generalized to the case of pseudoconvex programs. The concepts of pseudoconvexity and invexity do not depend on a single definition of the generalized directional derivative. 1. Introduction. Three characterizations of pseudoconvex functions are considered in this paper. The first is new. It is well-known that each pseudo- convex function is invex. Then the following question arises: what is the type of 2000 Mathematics Subject Classification: 26B25, 90C26, 26E15. Key words: Generalized convexity, nonsmooth function, generalized directional derivative, pseudoconvex function, quasiconvex function, invex function, nonsmooth optimization, solution sets, pseudomonotone generalized directional derivative. 204 Vsevolod Ivanov Ivanov the function η from the definition of invexity, when the invex function is pseudo- convex. This question is considered in Section 3, and a first order necessary and sufficient condition for pseudoconvexity of a function is given there. It is shown that the class of strongly pseudoconvex functions, considered by Weir [25], coin- cides with pseudoconvex ones. The main result of Section 3 is applied to characterize the solution set of a nonlinear programming problem in Section 4. The base results of Jeyakumar and Yang in the paper [13] are generalized there to the case, when the function is pseudoconvex.
    [Show full text]
  • Raport De Autoevaluare - 2012
    RAPORT DE AUTOEVALUARE - 2012 - 1. Date de identificare institut/centru : 1.1. Denumire: INSTITUTUL DE MATEMATICA OCTAV MAYER 1.2. Statut juridic: INSTITUTIE PUBLICA 1.3. Act de infiintare: Hotarare nr. 498 privind trecere Institutului de Matematica din Iasi la Academia Romana, din 22.02.1990, Guvernul Romaniei. 1.4. Numar de inregistrare in Registrul Potentialilor Contractori: 1807 1.5. Director general/Director: Prof. Dr. Catalin-George Lefter 1.6. Adresa: Blvd. Carol I, nr. 8, 700505-Iasi, Romania, 1.7. Telefon, fax, pagina web, e-mail: tel :0232-211150 http://www.iit.tuiasi.ro/Institute/institut.php?cod_ic=13. e-mail: [email protected] 2. Domeniu de specialitate : Mathematical foundations 2.1. Conform clasificarii UNESCO: 12 2.2. Conform clasificarii CAEN: CAEN 7310 (PE1) 3. Stare institut/centru 3.1. Misiunea institutului/centrului, directiile de cercetare, dezvoltare, inovare. Rezultate de excelenta in indeplinirea misiunii (maximum 2000 de caractere): Infiintarea institutului, in 1948, a reprezentat un moment esential pentru dezvoltarea, in continuare, a matematicii la Iasi. Cercetarile in prezent se desfasoara in urmatoarele directii: Ecuatii cu derivate partiale (ecuatii stochastice cu derivate partiale si aplicatii in studiul unor probleme neliniare, probleme de viabilitate si invarianta pentru ecuatii si incluziuni diferentiale si aplicatii in teoria controlului optimal, stabilizarea si controlabilitatea ecuatiilor dinamicii fluidelor, a sistemelor de tip reactie-difuzie, etc.), Geometrie (geometria sistemelor mecanice, geometria lagrangienilor pe fibrate vectoriale, structuri geometrice pe varietati riemanniene, geometria foliatiilor pe varietati semiriemanniene, spatii Hamilton etc), Analiza 1 matematica (analiza convexa, optimizare, operatori neliniari in spatii uniforme, etc.), Mecanica (elasticitate, termoelasticitate si modele generalizate in mecanica mediilor continue).
    [Show full text]
  • First Order Characterizations of Pseudoconvex Functions
    Serdica Math. J. 27 (2001), 203-218 FIRST ORDER CHARACTERIZATIONS OF PSEUDOCONVEX FUNCTIONS Vsevolod Ivanov Ivanov Communicated by A. L. Dontchev Abstract. First order characterizations of pseudoconvex functions are investigated in terms of generalized directional derivatives. A connection with the invexity is analysed. Well-known first order characterizations of the solution sets of pseudolinear programs are generalized to the case of pseudoconvex programs. The concepts of pseudoconvexity and invexity do not depend on a single definition of the generalized directional derivative. 1. Introduction. Three characterizations of pseudoconvex functions are considered in this paper. The first is new. It is well-known that each pseudo- convex function is invex. Then the following question arises: what is the type of 2000 Mathematics Subject Classification: 26B25, 90C26, 26E15. Key words: Generalized convexity, nonsmooth function, generalized directional derivative, pseudoconvex function, quasiconvex function, invex function, nonsmooth optimization, solution sets, pseudomonotone generalized directional derivative. 204 Vsevolod Ivanov Ivanov the function η from the definition of invexity, when the invex function is pseudo- convex. This question is considered in Section 3, and a first order necessary and sufficient condition for pseudoconvexity of a function is given there. It is shown that the class of strongly pseudoconvex functions, considered by Weir [25], coin- cides with pseudoconvex ones. The main result of Section 3 is applied to characterize the solution set of a nonlinear programming problem in Section 4. The base results of Jeyakumar and Yang in the paper [13] are generalized there to the case, when the function is pseudoconvex. The second and third characterizations are considered in Sections 5, 6.
    [Show full text]
  • Learning Output Kernels with Block Coordinate Descent
    Learning Output Kernels with Block Coordinate Descent Francesco Dinuzzo [email protected] Max Planck Institute for Intelligent Systems, 72076 T¨ubingen,Germany Cheng Soon Ong [email protected] Department of Computer Science, ETH Z¨urich, 8092 Z¨urich, Switzerland Peter Gehler [email protected] Max Planck Institute for Informatics, 66123 Saarbr¨ucken, Germany Gianluigi Pillonetto [email protected] Department of Information Engineering, University of Padova, 35131 Padova, Italy Abstract the outputs. In this paper, we introduce a method that We propose a method to learn simultane- simultaneously learns a vector-valued function and the ously a vector-valued function and a kernel kernel between the components of the output vector. between its components. The obtained ker- The problem is formulated within the framework of nel can be used both to improve learning per- regularization in RKH spaces. We assume that the formance and to reveal structures in the out- matrix-valued kernel can be decomposed as the prod- put space which may be important in their uct of a scalar kernel and a positive semidefinite kernel own right. Our method is based on the so- matrix that represents the similarity between the out- lution of a suitable regularization problem put components, a structure that has been considered over a reproducing kernel Hilbert space of in a variety of works, (Evgeniou et al., 2005; Capon- vector-valued functions. Although the regu- netto et al., 2008; Baldassarre et al., 2010). larized risk functional is non-convex, we show In practice, an important issue is the choice of the ker- that it is invex, implying that all local min- nel matrix between the outputs.
    [Show full text]
  • Invex Sets and Functions
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE Journal of Mathematical Analysis and Applications 263, 355–379 (2001) provided by Elsevier - Publisher Connector doi:10.1006/jmaa.2001.7574, available online at http://www.idealibrary.com on p r -Invex Sets and Functions Tadeusz Antczak Faculty of Mathematics, University of Łod´ ´z, Banacha 22, 90-238 Łod´ ´z, Poland E-mail: [email protected] Submitted by William F. Ames Received January 22, 2001 Notions of invexity of a function and of a set are generalized.The notion of an invex function with respect to η can be further extended with the aid of p-invex sets.Slight generalization of the notion of p-invex sets with respect to η leads to a new class of functions.A family of real functions called, in general, p r -pre- invex functions with respect to η (without differentiability) or p r -invex functions with respect to η (in the differentiable case) is introduced.Some (geometric) prop- erties of these classes of functions are derived.Sufficient optimality conditions are obtained for a nonlinear programming problem involving p r -invex functions with respect to η. 2001 Academic Press Key Words: p r -invex set with respect to η; r-invex set with respect to η; p r -pre-invex function with respect to ηp r -invex function with respect to η. 1.INTRODUCTION Convexity plays a vital role in many aspects of mathematical program- ming including, for example, sufficient optimality conditions and duality theorems.An invex function is one of the generalized convex functions
    [Show full text]
  • Arxiv:1608.04636V4 [Cs.LG]
    Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition Hamed Karimi, Julie Nutini, and Mark Schmidt Department of Computer Science, University of British Columbia Vancouver, British Columbia, Canada {[email protected]},{jnutini,schmidtm}@cs.ubc.ca Abstract. In 1963, Polyak proposed a simple condition that is sufficient to show a global linear convergence rate for gradient descent. This condition is a special case of theLojasiewicz inequality proposed in the same year, and it does not require strong convexity (or even convexity). In this work, we show that this much-older Polyak-Lojasiewicz (PL) inequality is actually weaker than the main conditions that have been explored to show linear convergence rates without strong convexity over the last 25 years. We also use the PL inequality to give new analyses of randomized and greedy coordinate descent methods, sign-based gradient descent methods, and stochastic gradient methods in the classic setting (with decreasing or constant step-sizes) as well as the variance- reduced setting. We further propose a generalization that applies to proximal-gradient methods for non-smooth optimization, leading to simple proofs of linear convergence of these methods. Along the way, we give simple convergence results for a wide variety of problems in machine learning: least squares, logistic regression, boosting, resilient backpropagation, L1-regularization, support vector machines, stochastic dual coordinate ascent, and stochastic variance-reduced gradient methods. 1 Introduction Fitting most machine learning models involves solving some sort of optimization problem. Gradient descent, and variants of it like coordinate descent and stochastic gradient, are the workhorse tools used by the field to solve very large instances of these problems.
    [Show full text]
  • Glimpses Upon Quasiconvex Analysis Jean-Paul Penot
    Glimpses upon quasiconvex analysis Jean-Paul Penot To cite this version: Jean-Paul Penot. Glimpses upon quasiconvex analysis. 2007. hal-00175200 HAL Id: hal-00175200 https://hal.archives-ouvertes.fr/hal-00175200 Preprint submitted on 27 Sep 2007 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ESAIM: PROCEEDINGS, Vol. ?, 2007, 1-10 Editors: Will be set by the publisher DOI: (will be inserted later) GLIMPSES UPON QUASICONVEX ANALYSIS Jean-Paul Penot Abstract. We review various sorts of generalized convexity and we raise some questions about them. We stress the importance of some special subclasses of quasiconvex functions. Dedicated to Marc Att´eia R´esum´e. Nous passons en revue quelques notions de convexit´eg´en´eralis´ee.Nous tentons de les relier et nous soulevons quelques questions. Nous soulignons l’importance de quelques classes particuli`eres de fonctions quasiconvexes. 1. Introduction Empires usually are well structured entities, with unified, strong rules (for instance, the length of axles of carts in the Chinese Empire and the Roman Empire, a crucial rule when building a road network). On the contrary, associated kingdoms may have diverging rules and uses.
    [Show full text]
  • Greed Is Good Greedy Optimization Methods for Large-Scale Structured Problems
    Greed is Good Greedy Optimization Methods for Large-Scale Structured Problems by Julie Nutini B.Sc., The University of British Columbia (Okanagan), 2010 M.Sc., The University of British Columbia (Okanagan), 2012 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in The Faculty of Graduate and Postdoctoral Studies (Computer Science) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) May 2018 c Julie Nutini 2018 The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Greed is Good: Greedy Optimization Methods for Large-Scale Structured Problems submitted by Julie Nutini in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science Examining Committee: Mark Schmidt, Computer Science Supervisor Chen Greif, Computer Science Supervisory Committee Member Will Evans, Computer Science Supervisory Committee Member Bruce Shepherd, Computer Science University Examiner Ozgur Yilmaz, Mathematics University Examiner ii Abstract This work looks at large-scale machine learning, with a particular focus on greedy methods. A recent trend caused by big datasets is to use optimization methods that have a cheap iteration cost. In this category are (block) coordinate descent and Kaczmarz methods, as the updates of these methods only rely on a reduced subspace of the problem at each iteration. Prior to our work, the literature cast greedy variations of these methods as computationally expensive with comparable convergence rates to randomized versions. In this dissertation, we show that greed is good. Specifically, we show that greedy coordinate descent and Kaczmarz methods have efficient implementations and can be faster than their randomized counterparts for certain common problem structures in machine learning.
    [Show full text]
  • Global Optimisation for Energy Systems
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The Australian National University Global Optimisation for Energy Systems Ksenia Bestuzheva October 2018 A thesis submitted for the degree of Doctor of Philosophy of The Australian National University © Copyright by Ksenia Bestuzheva 2018 All Rights Reserved Except where otherwise stated, this thesis is my own original work. Ksenia Bestuzheva October 2018 1 Abstract The goal of global optimisation is to find globally optimal solutions, avoiding local optima and other stationary points. The aim of this thesis is to provide more efficient global optimisation tools for energy systems planning and operation. Due to the ongoing increasing of complex- ity and decentralisation of power systems, the use of advanced mathematical techniques that produce reliable solutions becomes necessary. The task of developing such methods is com- plicated by the fact that most energy-related problems are nonconvex due to the nonlinear Alternating Current Power Flow equations and the existence of discrete elements. In some cases, the computational challenges arising from the presence of non-convexities can be tackled by relaxing the definition of convexity and identifying classes of problems that can be solved to global optimality by polynomial time algorithms. One such property is known as invexity and is defined by every stationary point of a problem being a global optimum. This thesis investigates how the relation between the objective function and the structure of the feasible set is connected to invexity and presents necessary conditions for invexity in the general case and necessary and sufficient conditions for problems with two degrees of freedom.
    [Show full text]
  • International Journal of Pure and Applied Mathematics ————————————————————————– Volume 15 No
    International Journal of Pure and Applied Mathematics ————————————————————————– Volume 15 No. 2 2004, 137-250 FUNDAMENTALS OF MIXED QUASI VARIATIONAL INEQUALITIES Muhammad Aslam Noor Etisalat College of Engineering P.O. Box 980, Sharjah, UNITED ARAB EMIRATES e-mail: [email protected] Abstract: Mixed quasi variational inequalities are very important and sig- nificant generalizations of variational inequalities involving the nonlinear bi- function. It is well-known that a large class of problems arising in various branches of pure and applied sciences can be studied in the general framework of mixed quasi variational inequalities. Due to the presence of the bifunction in the formulation of variational inequalities, projection method and its variant forms cannot be used to suggest and analyze iterative methods for mixed quasi variational inequalities. In this paper, we suggest and analyze various itera- tive schemes for solving these variational inequalities using resolvent methods, resolvent equations and auxiliary principle techniques. We discuss the sensitiv- ity analysis, stability of the dynamical systems and well-posedness of the mixed quasi variational inequalities. We also consider and investigate various classes of mixed quasi variational inequalities in the setting of invexity, g-convexity and uniformly prox-regular convexity. The concepts of invexity, g-convexity and uniformly prox-regular convexity are generalizations of the classical convexity in different directions. Some classes of equilibrium problems are introduced and studied. We also suggest several open problems with sufficient information and references. AMS Subject Classification: 49J40, 90C30, 35A15, 47H17 Key Words: mixed quasi variational inequalities, auxiliary principle, fixed- points, resolvent equations, iterative methods, convergence, dynamical systems, global convergence, stability Received: March 11, 2004 c 2004, Academic Publications Ltd.
    [Show full text]
  • Local Cone Approximations in Optimization
    Control and Cybernetics vol. 36 (2007) No. 3 Local cone approximations in optimization by M. Castellani1 and M. Pappalardo2 1 Department of “Sistemi ed Istituzioni per l’Economia” University of L’Aquila 2 Department of “Matematica Applicata” University of Pisa Abstract: We show how to use intensively local cone approxi- mations to obtain results in some fields of optimization theory such as optimality conditions, constraint qualifications, mean value theo- rems and error bound. Keywords: nonsmooth optimization, cone approximations, generalized directional epiderivatives, optimality conditions, con- straint qualifications, mean value theorem, error bound. 1. Introduction Many approaches have been developed in literature for obtaining results in op- timization theory. In this paper we propose a systematic approach to many problems of optimization theory (i.e. necessary and sufficient optimality con- ditions, constraint qualifications, mean value theorems and error bound) which uses local cone approximations of sets. This concept, introduced by Elster and Thierfelder (1988) and studied also in Castellani and Pappalardo (1995), appears to be very useful when used together with separation theorems and generalized derivatives. When intensively applied, we show that it is very flexible in our scope of interest. The first step of this scheme consists in approximating a set with a cone; in particular, when the set is the epigraph of a function, the cone represents the epigraph of a positively homogeneous function, which will be called generalized derivative. The subsequent step consists in considering the case in which the generalized derivative is the difference between two sublinear functions; finally, the most general situation, is the one in which the generalized derivative is a minimum of sublinear functions.
    [Show full text]
  • Learning Output Kernels with Block Coordinate Descent
    Learning output kernels with block coordinate descent Francesco Dinuzzo [email protected] Max Planck Institute for Intelligent Systems, 72076 T¨ubingen,Germany Cheng Soon Ong [email protected] Department of Computer Science, ETH Z¨urich, 8092 Z¨urich, Switzerland Peter Gehler [email protected] Max Planck Institute for Informatics, 66123 Saarbr¨ucken, Germany Gianluigi Pillonetto [email protected] Department of Information Engineering, University of Padova, 35131 Padova, Italy Abstract the outputs. In this paper, we introduce a method that We propose a method to learn simultane- simultaneously learns a vector-valued function and the ously a vector-valued function and a kernel kernel between the components of the output vector. between its components. The obtained ker- The problem is formulated within the framework of nel can be used both to improve learning per- regularization in RKH spaces. We assume that the formance and to reveal structures in the out- matrix-valued kernel can be decomposed as the prod- put space which may be important in their uct of a scalar kernel and a positive semidefinite kernel own right. Our method is based on the so- matrix that represents the similarity between the out- lution of a suitable regularization problem put components, a structure that has been considered over a reproducing kernel Hilbert space of in a variety of works, (Evgeniou et al., 2005; Capon- vector-valued functions. Although the regu- netto et al., 2008; Baldassarre et al., 2010). larized risk functional is non-convex, we show In practice, an important issue is the choice of the ker- that it is invex, implying that all local min- nel matrix between the outputs.
    [Show full text]