Everything Curl Megapack
Total Page:16
File Type:pdf, Size:1020Kb
Table of Contents Introduction 1.1 How to read this book 1.2 The cURL project 1.3 How it started 1.3.1 The name 1.3.2 What does curl do? 1.3.3 Project communication 1.3.4 Mailing list etiquette 1.3.5 Mailing lists 1.3.6 Reporting bugs 1.3.7 Releases 1.3.8 Security 1.3.9 Trust 1.3.10 The development team 1.3.11 Users of curl 1.3.12 Future 1.3.13 Get curl 1.4 Linux 1.4.1 Windows 1.4.2 macOS 1.4.3 Open Source 1.5 License 1.5.1 Copyright and Legal 1.5.2 Code of Conduct 1.5.3 Development 1.5.4 The source code 1.6 Code layout 1.6.1 Handling build options 1.6.2 Code style 1.6.3 Contributing 1.6.4 2 Reporting vulnerabilities 1.6.5 Web site 1.6.6 Building and installing 1.6.7 Build from source 1.6.7.1 Dependencies 1.6.7.2 TLS libraries 1.6.7.3 BoringSSL 1.6.7.3.1 Network and protocols 1.7 Networking simplified 1.7.1 Protocols 1.7.2 curl protocols 1.7.3 Command line basics 1.8 Command line options 1.8.1 Options depend on version 1.8.2 URLs 1.8.3 URL globbing 1.8.4 List options 1.8.5 Config file 1.8.6 Passwords 1.8.7 Progress meter 1.8.8 Using curl 1.9 Verbose 1.9.1 Trace options 1.9.1.1 Write out 1.9.1.2 Persistent connections 1.9.2 Downloads 1.9.3 Uploads 1.9.4 Connections 1.9.5 Timeouts 1.9.6 .netrc 1.9.7 Proxies 1.9.8 Exit status 1.9.9 FTP 1.9.10 Two connections 1.9.10.1 3 Directory traversing 1.9.10.2 Advanced FTP use 1.9.10.3 SCP and SFTP 1.9.11 IMAP and POP3 1.9.12 SMTP 1.9.13 TELNET 1.9.14 TLS 1.9.15 SSLKEYLOGFILE 1.9.15.1 Copy as curl 1.9.16 How to HTTP with curl 1.10 Protocol basics 1.10.1 Responses 1.10.2 Authentication 1.10.3 Ranges 1.10.4 HTTP versions 1.10.5 HTTPS 1.10.6 HTTP POST 1.10.7 Multipart formposts 1.10.8 -d vs -F 1.10.9 Redirects 1.10.10 Modify the HTTP request 1.10.11 HTTP PUT 1.10.12 Cookies 1.10.13 HTTP/2 1.10.14 Alternative Services 1.10.15 HTTP/3 1.10.16 HTTP cheat sheet 1.10.17 Scripting browser-like tasks 1.10.18 libcurl basics 1.11 Easy handle 1.11.1 Drive transfers 1.11.2 Drive with easy 1.11.2.1 Drive with multi 1.11.2.2 4 Drive with multi_socket 1.11.2.3 Connection reuse 1.11.3 Callbacks 1.11.4 Write data 1.11.4.1 Read data 1.11.4.2 Progress information 1.11.4.3 Header data 1.11.4.4 Debug 1.11.4.5 sockopt 1.11.4.6 SSL context 1.11.4.7 Seek and ioctl 1.11.4.8 Network data conversion 1.11.4.9 Opensocket and closesocket 1.11.4.10 SSH key 1.11.4.11 RTSP interleaved data 1.11.4.12 FTP matching 1.11.4.13 Cleanup 1.11.5 Name resolving 1.11.6 Proxies 1.11.7 Post transfer info 1.11.8 Share data between handles 1.11.9 URL API 1.11.10 API compatibility 1.11.11 --libcurl 1.11.12 Header files 1.11.13 Global initialization 1.11.14 multi-threading 1.11.15 curl easy options 1.11.16 TLS options 1.11.16.1 CURLcode return codes 1.11.17 Verbose operations 1.11.18 libcurl examples 1.11.19 Get a simple HTTP page 1.11.19.1 Get a page in memory 1.11.19.2 5 Submit a login form over HTTP 1.11.19.3 for C++ programmers 1.11.20 HTTP with libcurl 1.12 HTTP responses 1.12.1 HTTP requests 1.12.2 HTTP versions 1.12.3 HTTP ranges 1.12.4 HTTP authentication 1.12.5 Cookies with libcurl 1.12.6 Download 1.12.7 Upload 1.12.8 Bindings 1.13 libcurl internals 1.14 Index 1.15 6 Introduction Introduction Everything curl is an extensive guide to everything there is to know about curl, the project, the command-line tool, the library, how everything started and how it came to be what it is today. How we work on developing it further, what it takes to use it, how you can contribute with code and bug reports and why all those millions of existing users use it. This book is meant to be interesting and useful to both casual readers and the somewhat more experienced developers, and offers something for you all to pick and choose from. Do not read it from front to back. Read the chapters you are curious about and go back and forth as you see fit. I hope to run this book project as I do all other projects I work on: in the open, completely free to download and read, free for anyone to comment on, available for everyone to contribute to and help out with. Send your bug reports, pull requests or critiques to me and I will improve this book accordingly. This book will never be finished. I intend to keep working on it and while I may at some point consider it fairly complete and covering most aspects of the project (even if only that seems like an insurmountable goal), the curl project will continue to move so there will always be things to update in the book as well. This book project started at the end of September 2015. The book sites https://bookcurl.haxx.se is the home of this book. It features easy accessible links to read the book online in a web version or download a copy for offline reading using one of the many different versions offered, including PDF, ePUB and MOBI. https://ec.haxx.se is a shortcut to the HTML version of the book. https://github.com/bagder/everything-curl hosts all the book content. The author With the hope of becoming just a co-author of this material, I am Daniel Stenberg. I founded the curl project. I'm a developer at heart, for fun and profit. I live and work in Stockholm, Sweden. 7 Introduction All there is to know about me can be found on my web site. Help If you find mistakes, omissions, errors or blatant lies in this document, please send me a refreshed version of the affected paragraph and I will make amended versions. I will give proper credits to everyone who helps out! I hope to make this document better over time. Preferably, you submit errors or pull requests on the book's github page. Helpers Lots of people have reported bugs, improved sections or otherwise helped making this book what it is. These friends include the following: Anders Roxell, Angad Gill, Aris (Karim) Merchant, Ben Peachey, Carlton Gibson, Chris DeLuca, Citizen Esosa, Dan Fandrich, DrDoom74 at github, Emil Hessman, Frank Hassanabad, Gautham B A, Geir Hauge, Jay Satiro, JoyIfBam5, Luca Niccoli, Manuel, Marius Žilėnas, Martin van den Nieuwelaar, Ms2ger, Nick Travers, Oscar, Saravanan Musuwathi Kesavan, Senthil Kumaran, Viktor Szakats, Vitaliy T, Wieland Hoffmann, alawvt, bookofportals, ethomag, infinnovation-dev on github, mehandes License This document is licensed under the Creative Commons Attribution 4.0 license. 8 How to read this book How to read this book Here is an overview of the main sections of this book and what they cover. 1. The cURL project Project things. How it started, how we work and how often releases are made. 2. Open Source An attempt to explain what open source is and how it works. 3. The source code A description of the curl source tree and how the layout of the code is and works. 4. Network and protocols What exactly are networks and protocols? 5. Command line basics Start at the beginning. How do you use curl from a command line? 6. Using curl Going deeper, looking at things you do with curl the command line tool. 7. How to HTTP with curl Digging deeper on HTTP specific actions to do with the curl command line tool. 9 How to read this book 8. Building and installing Explaining how you can build curl and libcurl from source code. 9. libcurl basics How libcurl works and how you use it when writing your own applications with it. 10. HTTP with libcurl A closer look at doing HTTP specific things with libcurl. 11. Bindings A casual overview of some of the most popular libcurl bindings and how similar they are to the libcurl C API. 12. libcurl internals Under the hood it works like this… 13. Index The index. 10 The cURL project The cURL project A funny detail about Open Source projects is that they are called "projects", as if they were somehow limited in time or ever can get done. The cURL "project" is a number of loosely coupled individual volunteers working on writing software together with a common mission: to do reliable data transfers with Internet protocols. And giving away the code free for anyone to use.