DOCSLIB.ORG

File Transfer

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
File Transfer File Transfer Joon Kim PICSciE Research Computing Workshop Department of Chemistry 10/30/2017 Introduction Hyojoon (Joon) Kim. Ph.D. Cyber Infrastructure Engineer Role: Design & build a campus network and infrastructure that will better support our researchers 2 Our goal today • Overview of widely used tools • Learn about data transfer basics • Pick the right tool for your job • Know what to expect • Learn about Globus • Q&A 3 Data transfer Data Data source destination This part is our topic! 4 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. 2. At 2am, there was a brief 1-minute network outage. Transfer job aborted. 3. Arrive 8am in the morning. See the damage. Start again, which will take 10 hours. 4. Lost a day of work. Time Effort 5 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. Really? Are you SURE that’s the best? Time Effort 6 We want you to Focus on your research, not on transferring data around Time Effort 7 Data Transfer Tools 8 Transfer tools What transfer tool do you use? 9 What we normally see 10 Secure Copy (SCP) • Secure Copy (SCP) • Uses SSH for authentication and data transfer (TCP port 22) • Unix-based systems (including Mac OS X): Should have it by default • Windows: WinSCP (https://winscp.net/eng/download.php) 11 Secure Copy (SCP) • Secure Copy (SCP) • Uses SSH for authentication and data transfer (TCP port 22) • Unix-based systems (including Mac OS X): Should have it by default • Windows: WinSCP (https://winscp.net/eng/download.php) 12 rsync • rsync (rsync over SSH) • Sync files and directories between two endpoints. • Good for running backups. Careful with “--delete” option (this *mirrors* directories) • Unix-based systems (including Mac OS X): Should have it by default • Windows: CwRsync (https://itefix.net/cwrsync) 13 File Transfer Protocol (FTP) • ‘Secure’ File Transfer Protocol (‘S’FTP) • Widely used for file transfers • SFTP is more secure. Use it if available. • Unix-based systems (including Mac OS X): Should have it by default • Windows: FileZilla (https://filezilla-project.org) 14 File Transfer Protocol (FTP) • ‘Secure’ File Transfer Protocol (‘S’FTP) • Widely used for file transfers • SFTP is more secure. Use it if available. • Unix-based systems (including Mac OS X): Should have it by default • Windows: FileZilla (https://filezilla-project.org) 15 These tools are okay, but not always • Great compatibility. Widely available. • Small datasets. Quick transfers. (< 15 mins) • Large bulk data transfers. • Transfers on unreliable connections and hosts. 16 When should you look for other solutions? • Transfer is unreliable and takes a long time, to the point it affects your workflow • You are getting speeds less than (for large datasets): • Within campus • 800 Mbps (Mega bits per second). • 100 GB = ~820,000 Mb. Takes ~ 17 minutes • 3-4 Gbps if you have a 10G connection • 100 GB = ~800 Gb. Takes ~ 4 minutes • Between campus and outside • Hard to tell because of things out of our control • 200 Mbps – 5000 Mbps (5 Gbps) • 100 GB = ~820,000 Mb. Takes ~ 1 hour 17 Data Transfer Basics 18 Data transfer: Overview • The key players • Endpoints • Network 1/10/100 Gbps • Transfer tool Source Destination SCP FTP • Transfer settings SFTP rsync rsync over ssh Encrypted vs. not encrypted 19 (Why) is my data transfer slow? Where are the bottlenecks? scp scp ftp ftp Source Destination 20 1. Potential bottlenecks in endpoints • CPU and memory • Higher clock speed is better than # of cores • E.g., 2 x Intel Xeon® Broadwell processor E5-2643 3.4 GHz (total 12 cores) • RAM: 32 GB or more recommended • Disk I/O • Disk type (SATA HDD, SSD), configuration (RAID), and file system (ext4, GPFS, Lustre) • Decent server with HDD, EXT4 with RAID performs around 4 Gbps • RAID is required to get > 1 Gbps (ref: http://fasterdata.es.net/data-transfer-tools/) • Network Interface Controller (NIC) • Wireless: don’t expect much (mostly < 130 Mbps) • Wired: 1/10/40/100 Gbps • MiscellaneousYou tuning won’t get more than 1 Gb/s (125 MB/s) with • NIC tx buffer, enabling jumbo frames (9K instead of 1.5K), TCP/UDP tuning … your laptop, most desktops, and un-optimized servers 21 2. Potential bottlenecks in networks • Bandwidth • E.g., 1/10/40/100 Gb/s • Congestion • Time of day • Distance • E.g., Round Trip Time (RTT) between Stanford – Princeton: ~ 80ms • “Things” along the way • Routers, switches, firewalls, NAT, security devices, … 22 Our network, their network, and networks in between Parts we have limited visibility, and no control over 23 3. Transfer tools: Single vs multi stream Single stream - scp src dst - ftp - rsync Multi stream - Less packet loss (w/ dups) - Better utilization of link - GridFTP src dst - BBCP Faster transfer speed 24 Transfer tools: scp vs. GridFTP Downloading 500 GB data 8 hours 10 minutes (ref: http://fasterdata.es.net/data-transfer-tools/) 25 4. Transfer settings: Encryption Tool Encrypted Control Encrypted Data FTP HTTP (even password-based access) BBCP BBFTP ✔ Globus/GridFTP SCP SFTP rsync over SSH ✔ ✔ Globus/GridFTP with encryption-on HTTPS Data encryption provides best security, but negatively impacts transfer speed 26 In summary • Data transfer speed is affected by: Endpoints, network, transfer tool, and transfer settings • In most cases, your endpoint cannot handle much • Use wired connection, and check your bandwidth as far as you can • Ask for 10Gbps or 40Gbps connection if needed • Use better transfer tool if possible. • scp, (s)ftp, rsync, and wget/curl work fine for small transfers. • For large transfer (> GBs) over the WAN (RTT > 5ms: beyond Philly or NYC), don’t expect much from: scp, (s)ftp, rsync, wget/curl, robocopy 27 Hey… that’s a lot of work Transfer Endpoint Tool Transfer Network Settings Photo from Flickr (Billy Abbott) 28 What we have at Princeton: Data Transfer Nodes Test DTN Tigress DTN Lewis-Sigler PNI DTN CS DTN DTN Globus (GridFTP) 10 Gbps Data Transfer Nodes (DTNs) 10 Gbps (Mar. 2016 -) • High-end servers (Nov. 2015 -) • 10 Gbps connections ESnet • Tuned and optimized. RAID configured Internet2 • Good transfer tools (e.g., Globus) • Supported 29 Leverage our resources • Leverage our resources • Contact us or your departmental staff • About existing departmental DTNs and best work/dataflow • About having a departmental DTN 30 Data Transfer Nodes @ Princeton • Test DTN • /u/<NetID> • Contact: [email protected] • Tigress DTN (Princeton TIGRESS) • GPFS-based /tigress and scratch (tiger, della, orbital) disk space • Contact: [email protected] • LSI DTN (Lewis-Sigler Institute Core DTN) • LSI local cluster storage, lab data volumes, and scratch spaces • Contact: [email protected] • PNI DTN (Princeton Neuroscience Institute DTN) • All PNI `/Jukebox` Volumes (Bucket and Scratch spaces) are available • Contact: [email protected] • CS DTN (Computer Science Department DTN) • /n/fs/scratch/ and CS “project” storage spaces • Contact: [email protected] • Physics DTN (Princeton Physics DTN) • Restricted to users who has an account on Feynman cluster. /group, /scratch, and /mnt/<project> NFS files systems. • Contact: Vinod Gupta ([email protected]), Sumit Saluja ([email protected]) 31 What is Globus? https://www.globus.org • Fast, reliable data transfer and management service • Uses GridFTP underneath • Main advantages • Fast transfer speed (multi-stream) • Convenient to use: “Fire-and-Forget” 32 How it works 3. Data A B 1. Pick two endpoints 2. Submit transfer request at the Globus website 3. Dataset is transferred between two endpoints • Your machine’s web browser is just a “remote control” Globus.org • But, your machine can be an endpoint too (more later) 1. Select endpoints 4. Get 2. Request transfer notification 4. Get notification when transfer is done 33 What you need to use Globus • Account • Have Princeton NetID? You’re set. • Web browser • Chrome, Firefox, Safari, IE (Edge), etc • E.g., use smartphone to submit a transfer job (note: your phone is not transferring the dataset) • Access to source and destination endpoints 34 Your peers are actively using it! 09/2017: Total 257 transfers via Globus 35 Globus has good coverage • Universities • Cornell, NYU, Yale, Johns Hopkins, Dartmouth, Purdue, Georgia Tech, Virginia Tech, UVA, Michigan, Indiana, Stanford, Berkeley, and U Chicago,… • Most of the DOE national labs • ESnet at CERN, ANL, LBNL, LLNL, LANL, ORNL, and PNNL • National computing facilities • NERSC, NCSA, SDSC … • Federal agencies • NIH, USDA, NASA/JPL, USGS … • Over 50,000 registered endpoints at over 500 institutions worldwide 36 How to use it ’On’ by default. ‘On’ is around 25% slower than ‘Off’ ‘On’ is 10%-40% slower than ’Off’ 37 Globus Sharing • Share file or directory with other Globus users 38 Data Transfer via CLI • Install Globus CLI package (Linux or Cygwin on Windows) • https://docs.globus.org/cli/installation/ • Documentation • https://docs.globus.org/cli/ • Scheduling transfer jobs • $ echo "globus transfer $ep1:/share/godata/file1.txt $ep2:~/file1.txt -- label 'CLI Test Transfer 2’ " | at 13:07 oct 30 2017 • Cron job: 0 2 * * * runglobus.sh &> ~/runglobus.log 39 Globus Connect Personal https://www.globus.org/globus-connect-personal • Make your own machine a Globus endpoint • Mac, Windows, Linux • You are the administrator for your own Globus endpoint • Limited performance (# of streams), but convenient! 40 Use cases Scenario 1 • Data on your laptop • Globus Connect Personal Scenario 2 • Data on shared Windows PC e.g., • Globus Connect Personal Tigress DTN SMB mount Better for Scenario 2 • Data on shared Windows PC • SMB mount to Linux server • Globus Connect Server 41 Learn more about Globus • Globus documentation • https://docs.globus.org/how-to/ • Research Computing mini-course • “Transferring Large Data Sets, Plus Hands-on Tutorial with the Globus Transfer Tool” (Spring, 2018) • http://www.princeton.edu/researchcomputing/education/mini-courses/ 42 Other tools • BBCP • Free, easy to use, and comparable performance to Globus • Mac OS X, Linux-based systems.
Load more

Recommended publications
  • File Transfer: Basics and Best Practices
    File Transfer: Basics and Best Practices Joon Kim. Ph.D. PICSciE Research Computing Workshop @Chemistry 09/07/2018 Our goal today • Learn about data transfer basics • Pick the right tool for your job • Know what to expect • Overview of widely used tools • Learn about RC’s resources • Globus and Data Transfer Nodes • Q&A 2 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. 2. At 2am, there was a brief 1-minute network outage. Transfer job aborted. 3. Arrive 8am in the morning. See the damage. Start again, which will take 10 hours. Time 4. Lost a day of work. Effort 3 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. Is that really the best? Time Effort 4 We want you to Focus on your research, not on transferring data X X Time Effort 5 Use case 1 • I have data at Argonne National Lab that I want to process & analyze at Princeton HPC clusters Argonne’s Chemical Sciences and Engineering (CSE) division Princeton HPC Clusters 6 Use case 2 • I have data on my workstation/laptop that I want to process & analyze at Princeton HPC clusters workstation/laptop Princeton HPC Clusters 7 Data Transfer Basics 8 Data transfer: Overview Three key elements • Endpoints 1/10/100 Gbps • Network Source Destination SCP • Transfer tool FTP SFTP rsync rsync over ssh These will determine how you transfer data and how fast it will be 9 Why is my data transfer slow? Where are the bottlenecks? scp scp ftp ftp Source Destination 10 1.
    [Show full text]
  • Lftp Certificate Verification Not Trusted
    Lftp Certificate Verification Not Trusted Unversed Jed rupture navigably. If agential or alveated Mitchael usually spool his entirety wedging ravishingly or holings impavidly and unamusingly, how well-heeled is Constantin? Sometimes mock Yacov encircled her silks acquiescently, but self-repeating Tobit befriend unwieldily or debarks somnolently. The server is only providing the leaf certificate and not big chain. Lftp is a command-line file transfer program FTP client for UNIX and Unix-like systems. Lftp support ftps different from sftp ftps is ftp over the explicit TLS. Solution Nov 03 2020 Plesk does he manage FTPS mode settings implicit support explicit FTP. Vim lftprc set sslverify-certificate no step 2 lftp userwwwabccom Password userwwwabccom ls. 679902 ls Fatal error Certificate verification Not trusted. In lftp you edge the message Fatal error Certificate verification Not trusted It can consequence in adult directory see FILES section anon For implicit TLS SSL using lftp. Lftpftps GPJtag. Verification certificate not trusted WARNING Certificate verification unable to equip the first certificate. Sudden lftp problem poss authorisation related Networking. I do besides have a certificate specifically for this hostname so the verification fails. Lftp Kevin Bertoldi. LFTP as been compiled like real net-ftplftp-346 gnutls nls socks5 ssl Errors and issues. If your site will council allow keys you may be tedious to allow FTP from a fixed IP. From the compose page ftpssl-force boolean if for refuse and send password in point when server does water support SSL Default is air So their option isn't going. The web is ratio of suggestions to make this by disabling certificate verification or.
    [Show full text]
  • FTP Mini-HOWTO FTP Mini-HOWTO Table of Contents FTP Mini-HOWTO
    FTP mini-HOWTO FTP mini-HOWTO Table of Contents FTP mini-HOWTO.............................................................................................................................................1 Matthew Borowski ( [email protected] ) ( http://tarp.worldserve.net/ ).................................................1 1. Preamble..............................................................................................................................................1 1.1 Contacting Me....................................................................................................................................1 1.2 Legalities and Distribution.................................................................................................................1 1.3 History of this document....................................................................................................................1 2. About FTP............................................................................................................................................1 3. Beginner's guide to using ftp...............................................................................................................2 3.1 Running the ftp program....................................................................................................................2 3.2 Logging into an FTP server...............................................................................................................2 3.3 File transfer types...............................................................................................................................3
    [Show full text]
  • WIPO PCT Electronic Data Interchange (PCT-EDI) June 2020
    WIPO PCT Electronic Data Interchange (PCT-EDI) PCT Technical Cooperation June 2020 WIPO PCT Electronic Data Interchange (PCT-EDI) June 2020 Table of Contents I. INTRODUCTION ........................................................................................................... 2 II. SERVICE OVERVIEW ................................................................................................... 2 A. Why SSH? .................................................................................................................. 2 B. The WIPO Server ....................................................................................................... 3 C. Client Software ........................................................................................................... 3 III. OFFICE ACCOUNTS ................................................................................................. 3 A. Introduction ................................................................................................................ 3 B. Office account and Directory structure ........................................................................ 3 C. Maintenance ............................................................................................................... 4 D. Examples in this document ......................................................................................... 4 IV. CLIENT SOFTWARE CONFIGURATION ................................................................... 5 A. Introduction ...............................................................................................................
    [Show full text]
  • Linux Fundamentals (L-120)
    Linux Fundamentals (L-120) Modality: Virtual Classroom Duration: 5 Days About this course: The Linux Fundamentals course by QuickStart provides foundational knowledge about the workflow, concepts, and tools used in Linux and Unix. The importance of Linux cannot be overstated when it comes to network architecture and professional development, which is why this course is essential for individuals who want a better grasp over Linux, an operating system most people do not have experience in. The course will help beginners get comfortable with Unix systems and learn critical workflows about managing directories and files, using the vi editor, complying with the excellent security mechanisms built into Linux, and understand how data is processed and flows across Linux pipelines. The course also covers shell programs, how they are designed, how they are written, and how system engineers can increase efficiency by running concurrent processes. The average annual income of a Linux Systems Engineer is $102,367. Course Objectives: Completing the Linux Fundamentals course will give students knowledge in: Linus file manipulation Useful filesystem features I/O pipes and redirection Regular expressions and text manipulations Managing processes and jobs Vi, the Unix editor Task automation through shell scrips Software application management Remote administration Audience: This course is aimed at students and professionals who want to build a solid foundation in Linux and Unix, as well as professionals who want to hone their skills. The information taught in this course is applicable to Fedore Core, SUDE Linux Professional, Red Had Enterprise Linux, and SUSE Linus Enterprise Server. Prerequisites: This is a basic level course which only requires proficiency in using computers.
    [Show full text]
  • Data Transfer Basics and Best Practices
    Data Transfer Basics and Best Practices Joon Kim, Ph.D. PICSciE, Research Computing Research Computing Bootcamp 2018 10/29/2018 – 11/01/2018 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. 2. At 2am, there was a brief 1-minute network outage. Transfer job aborted. 3. Arrive 8am in the morning. See the damage. Start again, which will take 10 hours. Time 4. Lost a day of work. Effort 2 Why do we care? Without good practice, you will waste time and effort 1. Start data transfer using SCP at 10pm. Usually takes 10 hours. Is that really the best? Time Effort 3 We want you to Focus on your research, not on transferring data X X Time Effort 4 Use case 1 • I have data at Argonne National LaB that I want to process & analyze at Princeton HPC clusters Argonne National Lab Princeton RC HPC Clusters 5 Use case 2 • I have data on my workstation/laptop that I want to process & analyze at Princeton HPC clusters workstation/laptop Princeton RC HPC Clusters 6 Data Transfer Basics 7 Data transfer: Overview Three key elements • Endpoints 1/10/100 Gbps • Network Source Destination SCP • Transfer tool FTP SFTP rsync rsync over ssh These will determine how you transfer data and how fast it will be 8 Why is my data transfer slow? Where are the bottlenecks? scp scp ftp ftp Source Destination e.g., Server machine with 10 Gb/s connection will not guarantee 10 Gb/s transfer speed 9 Endpoint Examples SMB file mount NFS file mount Z: drive \\plabs.princeton.edu\data\ 10 1.
    [Show full text]
  • Linux Command Line Basics II: Downloading Data and Controlling Files Yanbin Yin
    Linux command line basics II: downloading data and controlling files Yanbin Yin 1 Things you should know about programming Learning programming has to go through the hands-on practice, a lot of practice Hearing what I describe about a command or a program helps, but you will not be able to do it unless you type in the codes and run it to see what happens Reading others’ codes helps but often is harder than writing it by yourself from scratch Although painful and frustrating, trouble-shooting is normal and part of the learning experience (ask experienced people or google) To avoid errors, you have to follow rules; most errors occurred in programming are because of not knowing rules or forgetting rules Use comments in case you forget what you’ve written means write-> run -> errors -> edit-> errors -> ………………………………….. -> run -> success Good news: finished scripts could be reused or edited for later use 2 Homework #7 1. Create a folder under your home called hw7 2. Change directory to hw7 3. Go to NCBI ftp site, find the genome, bacteria, ecoli MG1655 folder, and download the ptt file and the faa file in there 4. Create a copy of the ppt file, if the original file is called A.ptt, name the copied file A.ptt.bak Do the same thing for the faa file 5. Go to https://edu.isb-sib.ch/pluginfile.php/2878/mod_resource/content/4/couselab- html/content.html and finish all quizzes in Chapters 3 to 5. 6. Use what you learned in chapter 5 to count how many protein sequences are in the faa file of step 4.
    [Show full text]
  • Remote File Transfer Protocol by Using Multithreding
    BIOINFO Computer Engineering ISSN: 2249-3980 & E-ISSN: 2249-3999, Volume 2, Issue 1, 2012, pp.-31-35. Available online at http://www.bioinfo.in/contents.php?id=322 REMOTE FILE TRANSFER PROTOCOL BY USING MULTITHREDING GAMPAWAR A.D., BHAKTI JAIN, GABANE P. AND NARSWANI A.B. Department of Computer Science & Engineering, J.D.I.E.T., Yavatmal, India *Corresponding Author: Email- [email protected] Received: February 21, 2012; Accepted: March 15, 2012 Abstract- First module (FTP) of our framework would provide a user friendly FTP client application which would help even novice user to retrieve the file from remote destination server. Here users fall into two categories: A: Anonymous Users and B: Server Account Holder. In the first category the user can be anyone with no right to update, delete or upload the files on the server thus having limited capabilities. In case of the user having the account he/she has the privilege of writing, reading, update, delete or upload file on to the server. Following implementations would be involved in complete project – Creating a FTP Client Application with all functionalities and capabilities for executing various FTP commands and related functions. Config- uring DNS Server. This application provides a portal to the user through which the user has to specify the keywords along with the frequen- cy. The FTP Client will allow user to download, upload, update, delete and execute various FTP Commands. Keywords- FTP client, FTP server, DNS, TCP/IP Citation: Gampawar A.D., et al. (2012) Remote File Transfer Protocol by using Multithreding. BIOINFO Computer Engineering, ISSN: 2249- 3980 & E-ISSN: 2249-3999, Volume 2, Issue 1, pp.-31-35.
    [Show full text]
  • Getting Started with Unix.Pages
    Linux Tutorial Introduction Unix is an “old-school” operating system, developed at Bell Labs in the 1970s — before the mouse, menus, windows, Macs and PCs. Linux is an open-source version of Unix. Under Unix, you communicate with the computer by *typing commands*. This sounds primitive, but it has survived because it lends itself to automation. Typed commands can be put in a text file and executed as a “script”. Unix commands have short names — cd, pwd, ls — to save typing. These names are usually abbreviations: cd = Change Directory, pwd = Print Working Directory, ls = LiSt files. Unix operates under a philosophy of “no news is good news” and “no handholding the user”. You type a command and Unix does it, often silently. You can’t “see” that it did what you asked. If you type “rm *” (DON’T! — this deletes all files in the current folder) Unix complies, without asking “are you sure?”. bash The “shell” is the program you’re using when you’re not using a program. The default shell on our research computers is “bash”. Bash has many commands and features. You can write scripts in bash — short programs that automate repetitive tasks. man Help in Unix is primitive. It is vital to know the names of common commands. If you know the name, you can get help on how to use it by entering “man <command>”. (In these notes, <…> means “replace this with something relevant”.) Within man, spacebar or “ctrl-F” goes forward a page, “ctrl-B” backwards, and “q” quits. Man pages are hard to read.
    [Show full text]
  • Biowulf Communication Comparisons
    Biowulf batchlim show cpu and job limits for batch jobs freen show free and total nodes and cores jobdata show lots of info for a single jobid jobhist historical job summary jobload show current job and node status sacct select slurm jobs sbatch submit slurm job sbcast transmit a file to the nodes allocated to a Slurm job scancel delete slurm jobs seff display basic job accounting information sinfo view information about Slurm nodes and partitions sinteractive allocate an interactive session sjobs show brief summary of queued and running jobs squeue display status of slurm batch jobs sstat display various status information of a running job/step swarm submit a swarm of commands to cluster Communication clear clear the terminal screen curl transfer a URL dircolors color setup for ls ftp internet file transfer program glxinfo display information about OpenGL/GLX display lftp Sophisticated file transfer program lftpget same as lftp -c login sign on openssl OpenSSL command line tool rdist remote file distribution client program reset (see tset) resize set TERMCAP and terminal settings to current xterm window size rlogin remote login rsh remote shell screen screen manager with VT100/ANSI terminal emulation sendmail an electronic mail transport agent sftp secure file transfer program ssh OpenSSH SSH client (remote login program) telnet user interface to the TELNET protocol tset terminal initialization wall send a message to everybody's terminal wget The non-interactive network downloader xterm terminal emulator for X Comparisons bzcmp compare
    [Show full text]
  • FTP Mini-HOWTO
    FTP mini−HOWTO FTP mini−HOWTO Table of Contents FTP mini−HOWTO............................................................................................................................................1 Matthew Borowski ( [email protected] ) ( http://tarp.worldserve.net/ )................................................1 1. Preamble..............................................................................................................................................1 2. About FTP............................................................................................................................................1 3. Beginner's guide to using ftp...............................................................................................................1 4. Console FTP clients.............................................................................................................................1 5. X Window FTP clients........................................................................................................................1 6. FTP Servers..........................................................................................................................................2 1. Preamble..............................................................................................................................................2 1.1 Contacting Me....................................................................................................................................2 1.2 Legalities and Distribution.................................................................................................................2
    [Show full text]
  • Name Syntax Version Description
    lftp(1) General Commands Manual lftp(1) NAME lftp − Sophisticated file transfer program SYNTAX lftp [−d][−e cmd][−p port][−u user[,pass]] [site] lftp −f script_file lftp −c commands lftp −−version lftp −−help VERSION This man page documents lftp version 4.8.1. DESCRIPTION lftp is a file transfer program that allows sophisticated FTP,HTTP and other connections to other hosts. If site is specified then lftp will connect to that site otherwise a connection has to be established with the open command. lftp can handle several file access methods - FTP,FTPS, HTTP,HTTPS, HFTP,FISH, SFTP and file (HTTPS and FTPS are only available when lftp is compiled with GNU TLS or OpenSSL library). You can specify the method to use in ‘open URL’command, e.g. ‘open http://www.us.kernel.org/pub/linux’. HFTP is ftp-over-http-proxy protocol. It can be used automatically instead of FTP if ftp:proxy is set to ‘http://proxy[:port]’. Fish is a protocol working overanssh connection to a unix account. SFtp is a protocol implemented in SSH2 as SFTP subsystem. Besides FTP-likeprotocols, lftp has support for BitTorrent protocol as ‘torrent’ command. Seeding is also supported. Every operation in lftp is reliable, that is anynon-fatal error is handled properly and the operation is repeated. So if downloading breaks, it will be restarted from the point automatically.EvenifFTP server does not support the REST command, lftp will try to retrieve the file from the very beginning until the file is transferred completely. lftp has shell-likecommand syntax allowing you to launch several commands in parallel in background (&).
    [Show full text]