DOCSLIB.ORG

Table of Contents Transferring Files & Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Table of Contents Transferring Files & Data Table of Contents Transferring Files & Data.......................................................................................1 File Transfer: Overview......................................................................................................1 Verifying Files Transferred to the Lou Mass Storage System.............................................4 Using Shift for Local and Remote Transfers (Recommended)....................................5 Shift Transfer Tool Overview..............................................................................................5 Checking Shift Transfer Status and Restarting Transfers..................................................7 Shift Command Options.....................................................................................................9 Using Shift for Local Transfers and Tar Operations..........................................................19 Using Shift for Transfers and Tar Operations Between Two NAS Hosts...........................22 Using Shift for Remote Transfers and Tar Operations......................................................24 Local Transfers....................................................................................................27 Checking File Integrity.....................................................................................................27 Local File Transfer Commands.........................................................................................29 Shift Transfer Tool Overview............................................................................................31 Remote Transfers.................................................................................................32 Remote File Transfer Commands.....................................................................................32 Checking File Integrity.....................................................................................................36 Using GPG to Encrypt Your Data......................................................................................38 Shift Transfer Tool Overview............................................................................................41 Using bbFTP and bbSCP for Remote Transfers.................................................................42 The bbSCP Script........................................................................................................42 Using bbSCP for Test and Verification........................................................................45 Using bbFTP for Remote File Transfers......................................................................47 Using SUP for Remote Transfers......................................................................................52 Using the Secure Unattended Proxy (SUP).................................................................52 Advanced SUP Use.....................................................................................................58 Using the SUP Virtual File System........................................................................58 Using the SUP without the SUP Client...................................................................62 Examples.........................................................................................................................68 Inbound File Transfer Through SFEs: Examples.........................................................68 Outbound File Transfer Examples..............................................................................71 Optimizing/Troubleshooting..................................................................................73 Increasing File Transfer Rates..........................................................................................73 Dealing with Slow File Retrieval.......................................................................................74 TCP Performance Tuning for WAN Transfers....................................................................77 Optional Advanced Tuning for Linux................................................................................79 Streamlining PBS Job File Transfers to Lou......................................................................80 File Transfers Tips............................................................................................................81 Troubleshooting SCP File Transfer Failure with Protocol Error.........................................82 Transferring Files & Data File Transfer: Overview Here is a general overview of the various file transfer scenarios within the NAS environment, with pointers to related articles. File Transfers Between NAS Hosts For basic information about transferring files within the NAS secure enclave, see the following articles: • Local File Transfer Commands - cp, cxfscp, mcp, shiftc, rsync • Remote File Transfer Commands - scp, bbftp/bbscp, shiftc • Pleiades Front-End Usage Guidelines - file transfer between the compute systems or mass storage systems • Streamlining File Transfers from Pleiades Compute Nodes to Lou - within a PBS job • Checking File Integrity - to ensure the integrity of the data before and after the transfer File Transfers Between a NAS Host and Your Local Host Transferring files between a NAS host (such as Pleiades, Endeavour, or Lou) and a remote host, such as your local desktop, is more complex. There are multiple factors that you should be aware of: Which Commands to Use Remote file transfer commands such as scp, bbftp, bbscp, and sup/shiftc are supported on most NAS systems. Depending on the way the transfers are performed, you may need one or both of the client and server software for SCP, bbFTP, the bbSCP script, and/or the SUP/Shift client installed on your local host. Transfer Rates File transfer rates with the scp command, especially using SCP from versions of OpenSSH older than 4.7, can be as slow as 2 megabytes per second (MB/sec). For transferring large files over a long distance, consider doing the following: • Upgrade to the the latest version of OpenSSH. • Enable compression by adding -C to the scp command line if the data will compress well. • Use bbftp/bbscp and/or sup/shiftc. Security Issues • With SCP, your authentication information (such as password or passcode) and data are encrypted. • With bbFTP and bbSCP, only the authentication information is encrypted, while data is not. Transferring Files & Data 1 • You can use the SFEs or Secure Unattended Proxy (SUP). SUP is the easiest way to transfer files from and/or to your site if your local system is configured to allow the transfer. Inbound File Transfers When a file transfer command is initiated on a remote host, such as your local desktop, the transfer must go through either the SFEs or SUP. Using the Secure Front Ends Going through the SFEs requires authentication via RSA SecurID at the time of the transfer. You will be prompted for your RSA SecurID passcode when you issue the file transfer commands (such as scp, bbftp, or bbscp). You can use one of the following approaches: 1. One step via SSH Passthrough: Initiate scp, bbftp/bbscp from your local system to Pleiades, Endeavour, or Lou if SSH Passthrough has been set up. 2. One step via SSH ProxyCommand: Initiate scp from your local system through an SFE using the ssh ‑oProxyCommand option. To learn more, see Inbound File Transfers through SFEs Examples. Using the Secure Unattended Proxy Going through the SUP does not require RSA SecurID token authentication at the time of transfer. Instead, special "SUP keys" using RSA SecurID authentication must be obtained ahead of time. The SUP keys are good for one week and are used automatically to authenticate your file transfers using scp, bbftp, or bbscp issued on a command line or in a job script. TIP: We highly recommended that you learn about and use the Shift tool, which can be used together with the SUP to provide automated, reliable, and fast file transfers. WARNING: Although users have accounts on the SUP servers, no login session is allowed. File transfers going through SUP offers multiple benefits over going through the SFEs: • SUP allows transfers to be unattended; that is, you do not have to type in your password, passphrase, or passcode when the file transfer command is issued. So, file transfers can be done within a script that can be scheduled to run ahead of time. File transfers through the SFEs cannot be done in a script. • File transfers through SUP are done in one step, and setting up SSH passthrough is not needed since the SFEs are not involved. • SUP automatically sets some options, such as the port range allowed for bbFTP transfers, so that you don't have to set them explicitly. So, the syntax for bbFTP over SUP is greatly simplified compared to bbFTP without SUP. TIP: Some sites only allow specific outbound ports; SUP allows setting custom ports manually if needed. For example: sup bbftp -E 'bbftpd -e50000:51000' -e 'put foobar /tmp/foobar' pfe21.nas.nasa.gov See Using the Secure Unattended Proxy (SUP) and Shift Transfer Tool Overview for more information. File Transfer: Overview 2 NAS Username and Your Local Username If your NAS username and local username are different, you may have to add the appropriate username in the scp, bbftp, bbscp, or sup/shiftc command line. • If you issue the command on your local host, then username is your NAS username. • If you issue the command on a NAS host, then username is your local username. In the examples shown in Outbound File Transfer Examples and Inbound
Load more

Recommended publications
  • Administering Unidata on UNIX Platforms
    C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta UniData Administering UniData on UNIX Platforms UDT-720-ADMU-1 C:\Program Files\Adobe\FrameMaker8\UniData 7.2\7.2rebranded\ADMINUNIX\ADMINUNIXTITLE.fm March 5, 2010 1:34 pm Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Beta Notices Edition Publication date: July, 2008 Book number: UDT-720-ADMU-1 Product version: UniData 7.2 Copyright © Rocket Software, Inc. 1988-2010. All Rights Reserved. Trademarks The following trademarks appear in this publication: Trademark Trademark Owner Rocket Software™ Rocket Software, Inc. Dynamic Connect® Rocket Software, Inc. RedBack® Rocket Software, Inc. SystemBuilder™ Rocket Software, Inc. UniData® Rocket Software, Inc. UniVerse™ Rocket Software, Inc. U2™ Rocket Software, Inc. U2.NET™ Rocket Software, Inc. U2 Web Development Environment™ Rocket Software, Inc. wIntegrate® Rocket Software, Inc. Microsoft® .NET Microsoft Corporation Microsoft® Office Excel®, Outlook®, Word Microsoft Corporation Windows® Microsoft Corporation Windows® 7 Microsoft Corporation Windows Vista® Microsoft Corporation Java™ and all Java-based trademarks and logos Sun Microsystems, Inc. UNIX® X/Open Company Limited ii SB/XA Getting Started The above trademarks are property of the specified companies in the United States, other countries, or both. All other products or services mentioned in this document may be covered by the trademarks, service marks, or product names as designated by the companies who own or market them. License agreement This software and the associated documentation are proprietary and confidential to Rocket Software, Inc., are furnished under license, and may be used and copied only in accordance with the terms of such license and with the inclusion of the copyright notice.
    [Show full text]
  • CISCO-CONFIG-COPY-MIB: Secure Copy Support
    CISCO-CONFIG-COPY-MIB: Secure Copy Support The CISCO-CONFIG-COPY-MIB: Secure Copy Support feature enhances the CISCO-CONFIG-COPY-MIB by adding support for the copy Cisco IOS EXEC command, and implementing file transfers between a router and server using the secure copy protocol (scp). Feature Specifications for CISCO-CONFIG-COPY-MIB: Secure Copy Support Feature History Release Modification 12.3(2)T This feature was introduced. Supported Platforms Cisco 1710, Cisco 3600 series, Cisco 3725, Cisco 3745, Cisco 6400-NRP series, Cisco 7200, Cisco 7400, Cisco 7500, Cisco AS5300, Cisco AS5350, Cisco AS5400, Cisco AS5850, Cisco CVA 120, Cisco ICS 7750, Cisco ONS 15104, Cisco uBR 7200, Cisco uBR 925 Finding Support Information for Platforms and Cisco IOS Software Images Use Cisco Feature Navigator to find information about platform support and Cisco IOS software image support. Access Cisco Feature Navigator at http://www.cisco.com/go/fn. You must have an account on Cisco.com. If you do not have an account or have forgotten your username or password, click Cancel the login dialog box and follow the instructions that appear. Contents • , page 2 • How to Use Secure Copy Support, page 2 Configuration Examples for Secure Copy Support, page 3 Additional References, page 4 Command Reference, page 5 Americas Headquarters: Cisco Systems, Inc., 170 West Tasman Drive, San Jose, CA 95134-1706 USA © 2007 Cisco Systems, Inc. All rights reserved. CISCO-CONFIG-COPY-MIB: Secure Copy Support Information About CISCO-CONFIG-COPY-MIB Secure Copy Support Information About CISCO-CONFIG-COPY-MIB Secure Copy Support • • CISCO-CONFIG-COPY-MIB Secure Copy Implementation CISCO-CONFIG-COPY-MIB is platform-independent and provides objects to allow the copy functionality.
    [Show full text]
  • Install a VCS Release Key Via the Web Interface and CLI Configuration Example
    Install a VCS Release Key via the Web Interface and CLI Configuration Example Contents Introduction Prerequisites Requirements Components Used Configure Web Interface Release Key Installation Example CLI Release Key Installation Example Verify Web interface Verification of Release Key Installation CLI Interface Verification of Release Key Installation Troubleshoot Introduction This document describes the installation of a release key to a Cisco Video Communication Server (VCS) via the web interface and the Command Line Interface (CLI). Prerequisites Requirements Cisco recommends that you have knowledge of these topics: VCS Installation Have Installed successfully the VCS and applied a valid IP address that is reachable via web interface and or CLI. Have applied for and received a release key valid for the VCS serial number. Have access to the VCS with both root (by CLI) and the admin account by web interface or CLI. Have downloaded a VCS software upgrade image from Cisco.com. Note: Installation guides can be found here: http://www.cisco.com/c/en/us/support/unified- communications/telepresence-video-communication-server-vcs/products-installation-guides- list.html Components Used The information in this document is based on these software versions: VCS Version x8.6.1 and x8.7.3 VCS Control x7.X and x8.X releases VCS Expressway x7.X and x8.X releases PuTTY (terminal emulation software) ---Alternatively, you could use any terminal emulation software that supports SSH such as Secure CRT, TeraTerm and so on. PSCP (PuTTY Secure Copy Protocol client) ---You can use any client that supports SCP. Licensing email with a Release Key or Upgrade Key.
    [Show full text]
  • Pcoip Management Console 20.01 Administrators Guide
    Installing the PCoIP Management Console and Configuring Your System Installing the PCoIP Management Console and Configuring Your System The topics in this section contain information to help you get up and running quickly. Topics that refer to specific versions of PCoIP Management Console will be identified by the release number. Migrating, upgrading, or downgrading from other versions If you are migrating to a new PCoIP Management Console version see Migrating to a Newer Version. If you need to downgrade endpoints from firmware 5.0 or later to 4.8, see Downgrading Endpoints to Firmware 4.x. © 2020 Teradici 1 Installing PCoIP Management Console using vSphere Installing PCoIP Management Console using vSphere Once you have downloaded PCoIP Management Console, deploy it as an Open Virtual Appliance (OVA) using vSphere Client. To install PCoIP Management Console using vSphere Client: 1. Download the latest PCoIP Management Console OVA file to a location accessible from your vSphere Client. 2. Log in to your vSphere Client. 3. If you have more than one ESXi host, select the desired ESXi node; otherwise, there is no need to select a node. 4. From the vSphere client’s File menu, select Deploy OVF Template. 5. In the Source window, click Browse, select the PCoIP Management Console’s OVA file, click Open and Next. 6. In the OVF Template Details window, view the information and click Next. 7. In the End User License Agreement window, read the EULA information, click Accept and then Next. 8. In the Name and Location window, enter the name for your PCoIP Management Console and click Next.
    [Show full text]
  • OSI Model and Network Protocols
    CHAPTER4 FOUR OSI Model and Network Protocols Objectives 1.1 Explain the function of common networking protocols . TCP . FTP . UDP . TCP/IP suite . DHCP . TFTP . DNS . HTTP(S) . ARP . SIP (VoIP) . RTP (VoIP) . SSH . POP3 . NTP . IMAP4 . Telnet . SMTP . SNMP2/3 . ICMP . IGMP . TLS 134 Chapter 4: OSI Model and Network Protocols 4.1 Explain the function of each layer of the OSI model . Layer 1 – physical . Layer 2 – data link . Layer 3 – network . Layer 4 – transport . Layer 5 – session . Layer 6 – presentation . Layer 7 – application What You Need To Know . Identify the seven layers of the OSI model. Identify the function of each layer of the OSI model. Identify the layer at which networking devices function. Identify the function of various networking protocols. Introduction One of the most important networking concepts to understand is the Open Systems Interconnect (OSI) reference model. This conceptual model, created by the International Organization for Standardization (ISO) in 1978 and revised in 1984, describes a network architecture that allows data to be passed between computer systems. This chapter looks at the OSI model and describes how it relates to real-world networking. It also examines how common network devices relate to the OSI model. Even though the OSI model is conceptual, an appreciation of its purpose and function can help you better understand how protocol suites and network architectures work in practical applications. The OSI Seven-Layer Model As shown in Figure 4.1, the OSI reference model is built, bottom to top, in the following order: physical, data link, network, transport, session, presentation, and application.
    [Show full text]
  • Red Hat Jboss Data Grid 7.2 Data Grid for Openshift
    Red Hat JBoss Data Grid 7.2 Data Grid for OpenShift Developing and deploying Red Hat JBoss Data Grid for OpenShift Last Updated: 2019-06-10 Red Hat JBoss Data Grid 7.2 Data Grid for OpenShift Developing and deploying Red Hat JBoss Data Grid for OpenShift Legal Notice Copyright © 2019 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent.
    [Show full text]
  • K12522815: Modifying the BIG-IP Device SSL Certificate Configuration Using the Icontrol REST API
    K12522815: Modifying the BIG-IP device SSL certificate configuration using the iControl REST API Non-Diagnostic Original Publication Date: Apr 30, 2019 Update Date: Aug 6, 2021 Topic You want to use the iControl REST API to generate and apply a new self-signed SSL device certificate and key. You want to use the iControl REST API to apply an uploaded SSL certificate and key as the device certificate. Description You can use the iControl REST API to administer the SSL certificate and key that the Configuration utility uses. You can use the procedures in this article to generate a new self-signed certificate and key, which you can apply as the certificate and key used by the Configuration utility. Additionally, you can use a subset of the procedures to upload a certificate and key to the appropriate directories and then apply these as the certificate and key used by the Configuration utility. Typographic conventions The following typographic conventions are used in the command syntax examples: Note: If you are a new user of the iControl REST API, refer to K13225405: Common iControl REST API command examples. POST = curl -sk -u admin:<password> -H "Content-Type: application/json" -X POST https://<big-ip address> PUT = curl -sk -u admin:<password> -H "Content-Type: application/json" -X PUT https://<big-ip address> GET = curl -sk -u admin:<password> -H "Content-Type: application/json" -X GET https://<big-ip address> Prerequisites You must meet the following prerequisites to use this procedure: The BIG-IP system is licensed, provisioned, and configured with a management IP address.
    [Show full text]
  • P330-ML 4.5 RN.Fm
    Avaya P330-ML Version 4.5 Release Notes 1. Introduction This document contains information related to the Avaya P332G-ML, P332GT-ML and P334T-ML stackable switches that was not included in the User's Guide. This document also describes known issues, and other information required for proper installation and use of the product. 2. Important Notes • This software version is for P330-ML switches only. • You cannot stack P330-ML version 4.5 switches with P330 switches. • When you upgrade from version 3.x to version 4.5, you should first upgrade to version 4.0. Only then upgrade to 4.5. You can obtain firmware version 4.0 from www.avaya.com/support. • You must perform an NVRAM initialization before downloading module or stack configuration files, except for products that are configured with the factory settings. • P330-ML 4.5 Embedded Web Manager requires Java plug-in version 1.4.2. You may download this from the Avaya support site: www.avaya.com/support. — Please refer to the relevant Technical Note on the Avaya Support Site at www.avaya.com/support for managing Avaya products that require different Java plug-in versions. February 2004 1 3. What's New 3. What's New • Remote management access via SNMPv3 — SNMPv3 provides enhanced network management security with user- based authentication (SHA- or MD5-based), communication encryption (DES-based) and access control per-MIB item. • Support for both SNMPv3 and SNMPv2c traps. • SSH (Secure Shell) — SSH server functionality in the P330-ML provides enhanced remote session security using 3DES-CBC encryption, up to 2,048-bit DSA key and password-based user authentication.
    [Show full text]
  • Copyrighted Material
    Index Numerics Address Resolution Protocol (ARP), 1052–1053 admin password, SOHO network, 16-bit Windows applications, 771–776, 985, 1011–1012 900, 902 Administrative Tools window, 1081–1083, 32-bit (x86) architecture, 124, 562, 769 1175–1176 64-bit (x64) architecture, 124, 562, 770–771 administrative tools, Windows, 610 administrator account, 1169–1170 A Administrators group, 1171 ADSL (Asynchronous Digital Subscriber Absolute Software LoJack feature, 206 Line), 1120 AC (alternating current), 40 Advanced Attributes window, NTFS AC adapters, 311–312, 461, 468–469 partitions, 692 Accelerated Graphics Port (AGP), 58 Advanced Computing Environment (ACE) accelerated video cards (graphics initiative, 724 accelerator cards), 388 Advanced Confi guration and Power access points, wireless, 996, 1121 Interface (ACPI) standard, 465 access time, hard drive, 226 Advanced Graphics Port (AGP) card, access tokens, 1146–1147 391–392 Account Operators group, 1172 Advanced Graphics Port (AGP) port, 105 ACE (Advanced Computing Environment) Advanced Host Controller Interface (AHCI), initiative, 724 212–213 ACPI (Advanced Confi guration and Power Advanced Micro Devices (AMD), 141–144 Interface) standard, 465 Advanced Packaging Tool (APT), 572 Action Center, 1191–1192 Advanced Power Management (APM) Active Directory Database, 1145–1146, 1183 standard, 465 active heat sink, 150 Advanced Programmable Interrupt active matrix display, LCD (thin-fi lm Controller (APIC), 374 transistor (TFT) display), 470 Advanced RISC Computing Specifi cation active partition, 267,
    [Show full text]
  • Working with the Cisco IOS File System, Configuration Files, and Software Images
    APPENDIX B Working with the Cisco IOS File System, Configuration Files, and Software Images This appendix describes how to manipulate the Cisco ME 3400E Ethernet Access switch flash file system, how to copy configuration files, and how to archive (upload and download) software images to a switch. Note For complete syntax and usage information for the commands used in this chapter, see the switch command reference Cisco IOS Configuration Fundamentals Command Reference, Release 12.2 This appendix consists of these sections: • Working with the Flash File System, page B-1 Working with Configuration Files, page B-8 Working with Software Images, page B-23 Working with the Flash File System flash: Displaying Available File Systems, page B-2 Setting the Default File System, page B-3 Displaying Information about Files on a File System, page B-3 Creating and Removing Directories, page B-4 Copying Files, page B-4 Deleting Files, page B-5 Creating, Displaying, and Extracting tar Files, page B-6 Displaying the Contents of a File, page B-8 Cisco ME 3400E Ethernet Access Switch Software Configuration Guide OL-16485-01 B-1 Appendix B Working with the Cisco IOS File System, Configuration Files, and Software Images Working with the Flash File System Displaying Available File Systems show file systems privileged EXEC command as shown in this example. Switch# show file systems File Systems: Size(b) Free(b) Type Flags Prefixes * 15998976 5135872 flash rw flash: - - opaque rw bs: - - opaque rw vb: 524288 520138 nvram rw nvram: - - network rw tftp: - - opaque rw null: - - opaque rw system: - - opaque ro xmodem: - - opaque ro ymodem: Table B-1 show file systems Field Descriptions Field Value Size(b) Amount of memory in the file system in bytes.
    [Show full text]
  • TIP/Ix Utilities
    TIP/ix Utilities IP-617 December 2014 This edition applies to TIP/ix 2.5 and revision levels of TIP/ix 2.5 until otherwise indicated in a new edition. Publications can be requested from the address given below. Inglenet Business Solutions Inc reserves the right to modify or revise this document without notice. Except where a Software Usage Agreement has been executed, no contractual obligation between Inglenet Business Solutions Inc and the recipient is either expressed or implied. It is agreed and understood that the information contained herein is Proprietary and Confidential and that the recipient shall take all necessary precautions to ensure the confidentiality thereof. If you have a license agreement for TIP Studio or TIP/ix with Inglenet Business Solutions Inc, you may make copies of this documentation for internal use. Otherwise, you may not copy or transmit this document, in whole or in part, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of Inglenet Business Solutions Inc. Inglenet Business Solutions Inc Toll Free: 1-800-387-9391 Website: http://www.Inglenet.com Sales: [email protected] Help Desk: [email protected] TIP Studio, TIP/ix, and TIP/30, and are registered trade marks of Inglenet Business Solutions Inc: This documentation occasionally makes reference to the products of other corporations. These product names may be trade marks, registered or otherwise, or service marks of these corporations. Where this is the case, they are hereby acknowledged as such by Inglenet Business Solutions Inc. © Inglenet Business Solutions Inc, 1990-2014 TIP/ix Utility Programs Contents TIP/ix Utility Programs ................................................
    [Show full text]
  • The AWK Pattern Processing Language
    000.book.fm Page 531 Wednesday, September 30, 2009 10:13 AM 12 The AWK Pattern Processing Language In This Chapter AWK12Chapter12 is a pattern-scanning and processing language that searches one or more files for records (usually lines) that Syntax . 532 match specified patterns. It processes lines by performing Arguments . 532 actions, such as writing the record to standard output or Options . 533 incrementing a counter, each time it finds a match. Unlike procedural languages, AWK is data driven: You describe the Patterns . 534 data you want to work with and tell AWK what to do with Actions. 535 the data once it finds it. Variables . 535 You can use AWK to generate reports or filter text. It works Functions. 536 equally well with numbers and text; when you mix the two, Associative Arrays. 538 AWK usually comes up with the right answer. The authors of AWK (Alfred V. Aho, Peter J. Weinberger, and Brian W. Control Structures . 539 Kernighan) designed the language to be easy to use. To Examples . 541 achieve this end they sacrificed execution speed in the origi- getline: Controlling Input . 558 nal implementation. Coprocess: Two-Way I/O. 560 Getting Input from a Network. 562 Copyright © 2010 Mark G. Sobell 531531 000.book.fm Page 532 Wednesday, September 30, 2009 10:13 AM 532 Chapter 12 The AWK Pattern Processing Language AWK takes many of its constructs from the C programming language. It includes the following features: • A flexible format • Conditional execution • Looping statements • Numeric variables • String variables • Regular expressions • Relational expressions •C’s printf • Coprocess execution (gawk only) • Network data exchange (gawk only) Syntax A gawk command line has the following syntax: gawk [options] [program] [file-list] gawk [options] –f program-file [file-list] The gawk utility takes its input from files you specify on the command line or from standard input.
    [Show full text]