DOCSLIB.ORG

NSWI 0138: Advanced Unix Programming

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
NSWI 0138: Advanced Unix Programming NSWI 0138: Advanced Unix programming (c) 2011-2016 Vladim´ırKotal (c) 2009-2010 Jan Pechanec, Vladim´ırKotal SISAL MFF UK, Malostransk´en´am.25, 118 00 Praha 1 Charles University Czech Republic Vladim´ırKotal [email protected] March 10, 2016 1 Vladim´ırKotal NSWI 0138 (Advanced Unix programming) Contents 1 Overview 5 1.1 What is this lecture about? . .5 1.2 The lecture will cover... .5 1.3 A few notes on source code files . .6 2 Testing 6 2.1 Why?...........................................6 2.2 When ? . .6 2.3 Types of testing . .7 3 Debugging 8 3.1 Debuging in general . .8 3.2 Observing . .9 3.3 Helper tools . .9 3.3.1 ctags . .9 3.3.2 cscope . 10 3.3.3 OpenGrok . 11 3.3.4 Other tools . 11 3.4 Debugging data . 11 3.4.1 stabs . 11 3.4.2 DWARF . 12 3.4.3 CTF (Compact C Type Format) . 12 3.5 Resource leaks . 13 3.6 libumem . 13 3.6.1 How does libumem work . 14 3.6.2 Using libumem+mdb to find memory leaks . 14 3.6.3 How does ::findleaks work . 16 3.7 watchmalloc . 17 3.8 Call tracing . 18 3.9 Using /proc . 19 3.10 Debugging dynamic libraries . 20 3.11 Debuggers . 20 3.12 Symbol search and interposition . 20 3.13 dtrace . 21 4 Terminals 21 4.1 Terminal I/O Overview . 21 4.2 Terminal I/O Overview (cont.) . 22 4.3 Physical (Hardware) Terminal . 24 4.4 stty(1) command . 24 4.5 TTY Driver Connected To a Phy Terminal . 25 4.6 Pseudo terminals Overview . 26 4.7 PTY Driver Connected To a Process . 27 4.8 More Complex Example . 28 4.9 Reading From a Terminal . 29 4.10 Reading From a Terminal (cont.) . 30 4.11 Sessions and Process Groups (Jobs) . 30 4.12 Sessions and Process Groups (cont.) . 32 Page 2 of 87 Vladim´ırKotal NSWI 0138 (Advanced Unix programming) 4.13 Controlling Terminal . 33 4.14 Job Control . 34 4.15 What Happens If Job Control is Disabled? . 35 4.16 SIGHUP Signal . 35 4.17 So called hang-on-exit problem . 37 4.18 Function calls for sessions and process groups . 38 4.19 Opening a Terminal . 39 4.20 Working with Pseudoterminals . 39 4.21 Setting Terminal Attributes . 40 4.22 Terminfo database . 41 5 Advanced Network programming 42 5.1 Accessing low level structures . 42 5.2 More setsockopt() options . 43 5.3 Raw sockets . 44 5.4 Layer-2 (link) access . 45 5.5 Porting applications to IPv6 . 45 5.6 PF ROUTE socket . 47 5.7 PF KEY socket . 47 6 Advanced Thread Programming 48 6.1 Very Quick Recap of Threads . 49 6.2 Very Quick Recap of Threads (cont.) . 49 6.3 What we know (from POSIX API) . 50 6.4 Other Implementations . 50 6.5 Fork and Threads in POSIX . 51 6.6 Memory Leaks on fork(2) ............................... 52 6.7 Fork handlers - solution to fork-one safety issues . 52 6.8 Fork-Safe Library . 53 6.9 More on thread cancellation . 53 6.10 More on thread cancellation (cont.) . 54 6.11 Cancel-Safety in Libraries . 54 6.12 MT-Level Attribute in General . 55 6.13 Solaris Threads . 55 6.14 Solaris Threads API . 56 6.15 Differences between POSIX and Solaris Threads . 57 6.16 Combining both APIs . 57 6.17 Threads and Performance . 58 7 Advanced IPC and I/O 59 7.1 Advanced IPC and I/O . 59 7.2 Known Means of IPC . 59 7.3 ioctl(2) . 60 7.4 Example: /dev/crypto on Solaris . 60 7.5 Doors Overview . 61 7.6 Doors - how it works . 62 7.7 Sending file descriptors between processes . 63 7.8 Passing fd over a pipe (Solaris) . 63 7.9 Passing file descriptor over a socket . 63 7.10 Real-time signals - motivation . 64 Page 3 of 87 Vladim´ırKotal NSWI 0138 (Advanced Unix programming) 7.11 Using real-time signals . 64 7.12 Using real-time signals (continuted) . 65 7.13 Asynchronous versus Synchronous Programming . 65 7.14 Asynchronous I/O (AIO) { motivation . 66 7.15 Asynchronous POSIX I/O . 67 7.16 The aiocb structure . 67 7.17 Using POSIX Asynchronous I/O . 68 7.18 popen() and pclose() ................................ 68 7.19 popen() and pclose() example code . 69 7.20 Mechanisms for Creating a Process . 69 7.21 posix spawn(), posix spawnp() .......................... 70 7.22 Using posix spawn() ................................. 70 7.23 File actions . ..
Load more

Recommended publications
  • Digital Technical Journal, Number 3, September 1986: Networking
    Netwo;king Products Digital TechnicalJournal Digital Equipment Corporation Number 3 September I 986 Contents 8 Foreword William R. Johnson, Jr. New Products 10 Digital Network Architecture Overview Anthony G. Lauck, David R. Oran, and Radia J. Perlman 2 5 PerformanceAn alysis andModeling of Digital's Networking Architecture Raj Jain and William R. Hawe 35 The DECnetjSNA Gateway Product-A Case Study in Cross Vendor Networking John P:.. �orency, David Poner, Richard P. Pitkin, and David R. Oran ._ 54 The Extended Local Area Network Architecture and LANBridge 100 William R. Hawe, Mark F. Kempf, and Alan). Kirby 7 3 Terminal Servers on Ethernet Local Area Networks Bruce E. Mann, Colin Strutt, and Mark F. Kempf 88 The DECnet-VAXProduct -A n IntegratedAp proach to Networking Paul R. Beck and James A. Krycka 100 The DECnet-ULTRIXSoftware John Forecast, James L. Jackson, and Jeffrey A. Schriesheim 108 The DECnet-DOS System Peter 0. Mierswa, David). Mitton, and Ma�ha L. Spence 117 The Evolution of Network Management Products Nancy R. La Pelle, Mark). Seger, and Mark W. Sylor 129 The NMCCjDECnet Monitor Design Mark W. Sylor 1 Editor's Introduction The paper by Bill Hawe, Mark Kempf, and AI Kirby reports how studies of potential new broad­ band products led to the development of the Extended LAN Architecture. The design of the LANBridge 100, the first product incorporating that architecture, is described, along with the trade-offs made to achieve high performance. The speed of communication between terminals and systems depends on how they are connected. Bruce Mann, Colin Strutt, and Mark Kempf explain how they developed the LAT protocol to connect terminals to hosts on an Ethernet.
    [Show full text]
  • Solaris Und Opensolaris Eine Sinnvolle Alternative?
    Solaris und OpenSolaris Eine sinnvolle Alternative? Wolfgang Stief best Systeme GmbH MUCOSUG, GUUG e. V. [email protected] 2009-11-23 Agenda OpenSolaris, Solaris Express, Solaris Community Edition Das „Open“ in OpenSolaris Community, Lizenzen, Projects Features Container/Zones, ZFS, DTrace, Crossbow ... Und warum dann nicht gleich Linux? Solaris und OpenSolaris – eine sinnvolle Alternative? pg 2 OpenSolaris? Enterprise PowerNetwork ManagementVirtualization Installation Open Containers Storage CIFS Security Network- DTraceNetwork Based ZFS Auto- Packaging Predictive Magic Self Healing Hardware Time Optimizaton Slider Solaris und OpenSolaris – eine sinnvolle Alternative? pg 3 OpenSolaris Binary Distribution http://www.opensolaris.com stabiler, getester Code Support möglich erscheint ca. 2x jährlich, x86 und SPARC aktuelle Pakete (GNOME etc.), Installer neues Paketformat, Repositories ähnlich Debian aktuell (noch) 2009.06 Solaris und OpenSolaris – eine sinnvolle Alternative? pg 4 OpenSolaris Source Code http://www.opensolaris.org ab Juni 2005: zunächst DTrace, dann sukzessive weitere Teile aktive Community Source Browser OpenGrok http://src.opensolaris.org/ Features werden in „Projects“ entwickelt Community Release 2-wöchentlich (b127) „BFU“ nach Bedarf (blindingly fast upgrade) Solaris und OpenSolaris – eine sinnvolle Alternative? pg 5 OpenSolaris Community Launch am 14. Juni 2005 mehrere Distributionen aus der Community Schillix, Belenix, Nexenta, Milax, StormOS, OSUNIX Stand Frühjahr 2009 (ca. 3½ Jahre): → 116.000 registrierte Mitglieder
    [Show full text]
  • New/Usr/Src/Cmd/Mdb/Common
    new/usr/src/cmd/mdb/common/modules/genunix/findstack.c 1 new/usr/src/cmd/mdb/common/modules/genunix/findstack.c 2 ********************************************************** 642 return (DCMD_USAGE); 21346 Thu Feb 18 08:40:37 2016 new/usr/src/cmd/mdb/common/modules/genunix/findstack.c 644 if (interesting) { 6583 remove whole-process swapping 645 if (sobj != NULL || excl_sobj != NULL || ********************************************************** 646 tstate_str != NULL || excl_tstate_str != NULL) { ______unchanged_portion_omitted_ 647 mdb_warn( 648 "stacks: -i is incompatible with -[sStT]\n"); 583 /*ARGSUSED*/ 649 return (DCMD_USAGE); 584 int 650 } 585 stacks(uintptr_t addr, uint_t flags, int argc, const mdb_arg_t *argv) 651 excl_sobj = "CV"; 586 { 652 excl_tstate_str = "FREE"; 587 size_t idx; 653 } 589 char *seen = NULL; 655 if (caller_str != NULL) { 656 mdb_set_dot(0); 591 const char *caller_str = NULL; 657 if (mdb_eval(caller_str) != 0) { 592 const char *excl_caller_str = NULL; 658 mdb_warn("stacks: evaluation of \"%s\" failed", 593 uintptr_t caller = 0, excl_caller = 0; 659 caller_str); 594 const char *module_str = NULL; 660 return (DCMD_ABORT); 595 const char *excl_module_str = NULL; 661 } 596 stacks_module_t module, excl_module; 662 caller = mdb_get_dot(); 597 const char *sobj = NULL; 663 } 598 const char *excl_sobj = NULL; 599 uintptr_t sobj_ops = 0, excl_sobj_ops = 0; 665 if (excl_caller_str != NULL) { 600 const char *tstate_str = NULL; 666 mdb_set_dot(0); 601 const char *excl_tstate_str = NULL; 667 if (mdb_eval(excl_caller_str) !=
    [Show full text]
  • Pexpect Documentation Release 4.8
    Pexpect Documentation Release 4.8 Noah Spurrier and contributors Apr 17, 2021 Contents 1 Installation 3 1.1 Requirements...............................................3 2 API Overview 5 2.1 Special EOF and TIMEOUT patterns..................................6 2.2 Find the end of line – CR/LF conventions................................6 2.3 Beware of + and * at the end of patterns.................................7 2.4 Debugging................................................8 2.5 Exceptions................................................8 2.6 Pexpect on Windows...........................................9 3 API documentation 11 3.1 Core pexpect components........................................ 11 3.2 fdpexpect - use pexpect with a file descriptor.............................. 23 3.3 popen_spawn - use pexpect with a piped subprocess.......................... 23 3.4 replwrap - Control read-eval-print-loops................................. 24 3.5 pxssh - control an SSH session...................................... 25 4 Examples 33 5 FAQ 35 6 Common problems 39 6.1 Threads.................................................. 39 6.2 Timing issue with send() and sendline()................................. 39 6.3 Truncated output just before child exits................................. 40 6.4 Controlling SSH on Solaris....................................... 40 6.5 child does not receive full input, emits BEL............................... 40 7 History 41 7.1 Releases................................................. 41 7.2 Moves and forks............................................
    [Show full text]
  • Ajuba Solutions Version 1.4 COPYRIGHT Copyright © 1998-2000 Ajuba Solutions Inc
    • • • • • • Ajuba Solutions Version 1.4 COPYRIGHT Copyright © 1998-2000 Ajuba Solutions Inc. All rights reserved. Information in this document is subject to change without notice. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means electronic or mechanical, including but not limited to photocopying or recording, for any purpose other than the purchaser’s personal use, without the express written permission of Ajuba Solutions Inc. Ajuba Solutions Inc. 2593 Coast Avenue Mountain View, CA 94043 U.S.A http://www.ajubasolutions.com TRADEMARKS TclPro and Ajuba Solutions are trademarks of Ajuba Solutions Inc. Other products and company names not owned by Ajuba Solutions Inc. that appear in this manual may be trademarks of their respective owners. ACKNOWLEDGEMENTS Michael McLennan is the primary developer of [incr Tcl] and [incr Tk]. Jim Ingham and Lee Bernhard handled the Macintosh and Windows ports of [incr Tcl] and [incr Tk]. Mark Ulferts is the primary developer of [incr Widgets], with other contributions from Sue Yockey, John Sigler, Bill Scott, Alfredo Jahn, Bret Schuhmacher, Tako Schotanus, and Kris Raney. Mark Diekhans and Karl Lehenbauer are the primary developers of Extended Tcl (TclX). Don Libes is the primary developer of Expect. TclPro Wrapper incorporates compression code from the Info-ZIP group. There are no extra charges or costs in TclPro due to the use of this code, and the original compression sources are freely available from http://www.cdrom.com/pub/infozip or ftp://ftp.cdrom.com/pub/infozip. NOTE: TclPro is packaged on this CD using Info-ZIP’s compression utility.
    [Show full text]
  • Systems Programming II
    Systems Programming II Iqbal Mohomed CSC 209 – Summer 2004 Week 7 Motivation for Signals • When a program forks into 2 or more processes, rarely do they execute independently of each other • The processes usually require some form of synchronization, and this is typically handled using signals. Job control is another important use • Data usually needs to be passed between processes also, and this is typically handled using pipes and sockets, which we will discuss shortly • Signals are usually generated by – Machine interrupts – The program itself, other programs, or the user (i.e. from the keyboard) Introduction to Signals • When a C program receives a signal, control is immediately passed to a function called a signal handler • The signal handler function can execute some C statements and exit in three different ways: – Return control to the place in the program which was executing when the signal occurred – Return control to some other point in the program – Terminate the program by calling the exit (or _exit) function signal() • A default action is provided for each kind of signal, such as terminate, stop or ignore • For nearly all signal types, the default action can be changed using the signal() function. The exceptions are SIGKILL and SIGSTOP. The handler is defined as follows: – typedef void (*sighandler_t)(int); • To change the handler: – sighandler_t signal(int signum, sighandler_t handler); More on signal() • For each process, the OS maintains a table of actions that should be performed for each kind of signal. The signal() function changes the table entry for the signal named as the first argument to the value provided as the second argument.
    [Show full text]
  • Ubuntu Kung Fu
    Prepared exclusively for Alison Tyler Download at Boykma.Com What readers are saying about Ubuntu Kung Fu Ubuntu Kung Fu is excellent. The tips are fun and the hope of discov- ering hidden gems makes it a worthwhile task. John Southern Former editor of Linux Magazine I enjoyed Ubuntu Kung Fu and learned some new things. I would rec- ommend this book—nice tips and a lot of fun to be had. Carthik Sharma Creator of the Ubuntu Blog (http://ubuntu.wordpress.com) Wow! There are some great tips here! I have used Ubuntu since April 2005, starting with version 5.04. I found much in this book to inspire me and to teach me, and it answered lingering questions I didn’t know I had. The book is a good resource that I will gladly recommend to both newcomers and veteran users. Matthew Helmke Administrator, Ubuntu Forums Ubuntu Kung Fu is a fantastic compendium of useful, uncommon Ubuntu knowledge. Eric Hewitt Consultant, LiveLogic, LLC Prepared exclusively for Alison Tyler Download at Boykma.Com Ubuntu Kung Fu Tips, Tricks, Hints, and Hacks Keir Thomas The Pragmatic Bookshelf Raleigh, North Carolina Dallas, Texas Prepared exclusively for Alison Tyler Download at Boykma.Com Many of the designations used by manufacturers and sellers to distinguish their prod- ucts are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf and the linking g device are trademarks of The Pragmatic Programmers, LLC.
    [Show full text]
  • Practical C Programming, 3Rd Edition
    Practical C Programming, 3rd Edition By Steve Oualline 3rd Edition August 1997 ISBN: 1-56592-306-5 This new edition of "Practical C Programming" teaches users not only the mechanics or programming, but also how to create programs that are easy to read, maintain, and debug. It features more extensive examples and an introduction to graphical development environments. Programs conform to ANSI C. 0 TEAM FLY PRESENTS Table of Contents Preface How This Book is Organized Chapter by Chapter Notes on the Third Edition Font Conventions Obtaining Source Code Comments and Questions Acknowledgments Acknowledgments to the Third Edition I. Basics 1. What Is C? How Programming Works Brief History of C How C Works How to Learn C 2. Basics of Program Writing Programs from Conception to Execution Creating a Real Program Creating a Program Using a Command-Line Compiler Creating a Program Using an Integrated Development Environment Getting Help on UNIX Getting Help in an Integrated Development Environment IDE Cookbooks Programming Exercises 3. Style Common Coding Practices Coding Religion Indentation and Code Format Clarity Simplicity Summary 4. Basic Declarations and Expressions Elements of a Program Basic Program Structure Simple Expressions Variables and Storage 1 TEAM FLY PRESENTS Variable Declarations Integers Assignment Statements printf Function Floating Point Floating Point Versus Integer Divide Characters Answers Programming Exercises 5. Arrays, Qualifiers, and Reading Numbers Arrays Strings Reading Strings Multidimensional Arrays Reading Numbers Initializing Variables Types of Integers Types of Floats Constant Declarations Hexadecimal and Octal Constants Operators for Performing Shortcuts Side Effects ++x or x++ More Side-Effect Problems Answers Programming Exercises 6.
    [Show full text]
  • Version 7.8-Systemd
    Linux From Scratch Version 7.8-systemd Created by Gerard Beekmans Edited by Douglas R. Reno Linux From Scratch: Version 7.8-systemd by Created by Gerard Beekmans and Edited by Douglas R. Reno Copyright © 1999-2015 Gerard Beekmans Copyright © 1999-2015, Gerard Beekmans All rights reserved. This book is licensed under a Creative Commons License. Computer instructions may be extracted from the book under the MIT License. Linux® is a registered trademark of Linus Torvalds. Linux From Scratch - Version 7.8-systemd Table of Contents Preface .......................................................................................................................................................................... vii i. Foreword ............................................................................................................................................................. vii ii. Audience ............................................................................................................................................................ vii iii. LFS Target Architectures ................................................................................................................................ viii iv. LFS and Standards ............................................................................................................................................ ix v. Rationale for Packages in the Book .................................................................................................................... x vi. Prerequisites
    [Show full text]
  • Name Description Files Notes See Also Colophon
    PTS(4) Linux Programmer’sManual PTS(4) NAME ptmx, pts − pseudoterminal master and slave DESCRIPTION The file /dev/ptmx is a character file with major number 5 and minor number 2, usually with mode 0666 and ownership root:root. It is used to create a pseudoterminal master and slave pair. When a process opens /dev/ptmx,itgets a file descriptor for a pseudoterminal master (PTM), and a pseu- doterminal slave (PTS) device is created in the /dev/pts directory.Each file descriptor obtained by opening /dev/ptmx is an independent PTM with its own associated PTS, whose path can be found by passing the file descriptor to ptsname(3). Before opening the pseudoterminal slave,you must pass the master’sfile descriptor to grantpt(3) and un- lockpt(3). Once both the pseudoterminal master and slave are open, the slave provides processes with an interface that is identical to that of a real terminal. Data written to the slave ispresented on the master file descriptor as input. Data written to the master is presented to the slave asinput. In practice, pseudoterminals are used for implementing terminal emulators such as xterm(1), in which data read from the pseudoterminal master is interpreted by the application in the same way a real terminal would interpret the data, and for implementing remote-login programs such as sshd(8), in which data read from the pseudoterminal master is sent across the network to a client program that is connected to a terminal or terminal emulator. Pseudoterminals can also be used to send input to programs that normally refuse to read input from pipes (such as su(1), and passwd(1)).
    [Show full text]
  • Unix System Programming Overview Outline What Is a Signal? Signal
    Overview Last Week: ● How to program UNIX processes (Chapters 7-9) ● fork() and exec() Unix System Programming This Week, and next week: ● UNIX inter-process communication mechanisms: signals, Signals » (next week) pipes and FIFOs. ● How to program with UNIX signals (Chapter 10) » http://en.wikipedia.org/wiki/Unix_signal ● Non-local jumps (Chapter 7) ● Focus on the sigaction() function Maria Hybinette, UGA 1 Maria Hybinette, UGA 2 Outline What is a Signal? ● A signal is an asynchronous event which is ● What is a UNIX signal? delivered to a process (instantiated by a small message) ● Signal types ● Asynchronous means that the event can occur ● Generating signals at any time (e.g., posting at a bulletin board ) ● Responding to a signal » may be unrelated to the execution of the process ● Common uses of a signal – e.g., user types Ctrl-C, or the modem hangs (SIGINT) ● Implementing a read() time-out – e.g,, user types Ctrl-Z (SIGTSTP) ● Sent from kernel (e.g. detects divide by zero ● Non-local jumps setjmp()/longjmp() (SIGFPE) or could be at the request of another ● POSIX signals process to send to another) ● Interrupted system calls ● Only information that a signal carries is its ● System calls inside handlers unique ID and that it arrived Maria Hybinette, UGA 3 Maria Hybinette, UGA 4 Signal Types (31 in POSIX) Signal Sources terminal memory ID Name Description Default Action driver management shell command 2 SIGINT Interrupt from keyboard (^C) terminate Ctr-C SIGINT SIGHUP 3 SIGQUIT Quit from keyboard (^\) terminate & core SIGSEGV 9 SIGKILL
    [Show full text]
  • Solaris 10 End of Life
    Solaris 10 end of life Continue Oracle Solaris 10 has had an amazing OS update, including ground features such as zones (Solaris containers), FSS, Services, Dynamic Tracking (against live production operating systems without impact), and logical domains. These features have been imitated in the market (imitation is the best form of flattery!) like all good things, they have to come to an end. Sun Microsystems was acquired by Oracle and eventually, the largest OS known to the industry, needs to be updated. Oracle has set a retirement date of January 2021. Oracle indicated that Solaris 10 systems would need to raise support costs. Oracle has never provided migratory tools to facilitate migration from Solaris 10 to Solaris 11, so migration to Solaris has been slow. In September 2019, Oracle decided that extended support for Solaris 10 without an additional financial penalty would be delayed until 2024! Well its March 1 is just a reminder that Oracle Solaris 10 is getting the end of life regarding support if you accept extended support from Oracle. Combined with the fact gdpR should take effect on May 25, 2018 you want to make sure that you are either upgraded to Solaris 11.3 or have taken extended support to obtain any patches for security issues. For more information on tanningix releases and support dates of old and new follow this link ×Sestive to abort the Unix Error Operating System originally developed by Sun Microsystems SolarisDeveloperSun Microsystems (acquired by Oracle Corporation in 2009)Written inC, C'OSUnixWorking StateCurrentSource ModelMixedInitial release1992; 28 years ago (1992-06)Last release11.4 / August 28, 2018; 2 years ago (2018-08-28)Marketing targetServer, PlatformsCurrent: SPARC, x86-64 Former: IA-32, PowerPCKernel typeMonolithic with dynamically downloadable modulesDefault user interface GNOME-2-LicenseVariousOfficial websitewww.oracle.com/solaris Solaris is the own operating system Of Unix, originally developed by Sunsystems.
    [Show full text]