(on Wikipedia)
http://lftp.yar.ru/ [ 1 ]
An FTP-like program which can also Mirror.
I use this to synchronize files over SFTP (SSH).
--
- 2020-03-25 - Some time ago I switched to using rsync over SSH.
- 2020-03-25 - 4.8.4-2 on Debian 10.1.0-amd64-xfce-CD-1
- 2020-03-25 - 4.7.4-1 on Debian 9.9.0-i386-xfce-CD-1
-
2016-04-10 - 4.4.9 on Slackware 14.1
- Libraries used: Readline 5.2, Expat 2.1.0, OpenSSL 1.0.1s 1 Mar 2016, zlib 1.2.8
-
2016-03-26 - 4.4.13 on Lubuntu 14.04.4 LTS
- Libraries used: Readline 6.3, GnuTLS 2.12.23, zlib 1.2.8
- 1 Intro/Notes
- 2 LFTP for BitTorrent
- 3 Usage
- 3.1 Mirroring (download remote files TO your local machine)
- 3.2 Reverse-mirroring (push local files TO the remote FTP site)
- 3.3 Active Mode
- 3.4 Regular expressions
- 3.5 FXP (transferring directly FROM one remote FTP server TO another remote FTP server)
- 3.6 Multiple Commands
- 3.7 Find content in a file
- 3.8 Leeching a website
- 4 Troubleshooting
- 5 2010-02-02
- 6 2006-12-21
Intro/Notes ∞
Has some spectacular features:
- Mirroring and "reverse mirroring"
- directory listing caching
- colour lists
- find, grep,
ls -al
- job control with ^z, fg, etc.
!ls
for local listing.!<command>
for whatever, locally.- To get into an sftp site with LFTP, do
lftp sftp://username@host
-
lcd <path>
to locallycd
and stay there for the purposes of file transfers.
LFTP for BitTorrent ∞
First explored on 2013-06-23 with BitTorrent with the Raspberry Pi
- No encryption support.
- No ability to select individual files to download.
- No ability to set individual file priority.
-
Cannot specify/modify a list of trackers. I assume it gets something from a .torrent file, if given one.
- Yes
.torrent
files and trackers are technically obsolete, but they still have some value.
- Yes
I didn't even research multiple torrents. It's a safe bet that there's no prioritization between them. Multiple LFTP torrent processes fighting it out does not sound like a good idea.
Usage ∞
There are scripts scattered throughout https://github.com/spiralofhope/shell-random/tree/master.
You would be better off getting the whole repository, then using its findinall()
for \lftp
(including that backslash).
Mirroring (download remote files TO your local machine) ∞
set ftp:list-options -a
- Also download files which start with a dot.--parallel=2
- Download two files in parallel. I like the idea of flooding my connection to help the overall download speed stay high.-v
- verbose. Shouldn't this be a default? Sigh..-
-e
- delete. Replicate deletions.
set ftp:list-options -a mirror --parallel=2 -v
Reverse-mirroring (push local files TO the remote FTP site) ∞
set ftp:list-options -a
- Also download files which start with a dot.--parallel=2
- Download two files in parallel. I like the idea of flooding my connection to help the overall download speed stay high.-v
- verbose. Shouldn't this be a default? Sigh..-R
- reverse. Sends files to the remote location.-
-e
- delete. Replicate deletions.
set ftp:list-options -a mirror --parallel=2 -Rv
- issues
- 2008-06-20 -- I've had the FTP connection hang repeatedly and fail on all retries.
-
2008-06-20 -- I've tried re-sending things, and saw it remove and re-send a significant number of files.
Active Mode ∞
-
2006-07-20 -- I don't know why I can't just switch between modes while connected, or why I can't use a switch on the commandline when running LFTP. Presumably because humans didn't make this program:
I have to edit $HOME/.lftp/rc
, which doesn't exist so I don't know what to put in it.
The documentation says a bunch of shit which doesn't mean anything to me. There is no example.
ftp:passive-mode (boolean) sets passive ftp mode. This can be useful if you are behind a firewall or a dumb masquerading router. In passive mode lftp uses PASV command, not the PORT com- mand which is used in active mode. In passive mode lftp itself makes the data connection to the server; in active mode the server connects to lftp for data trans- fer. Passive mode is the default.
I think set ftp:passive-mode false
does do something, but Googling gives me better documentation:
set passive-mode/ftp.site1.com yes set passive-mode/ftp.site2.org no set passive-mode/*.site3.net yes
2007-07-09 -- This was tested and works. If $HOME/.lftp/rc
doesn't exist, make it.
Regular expressions ∞
LFTP does it like grep, so you're out of luck if you want to do it in an easy way.
exclude:
-x '^certs' -x '^store'
Like so:
set ftp:list-options -a mirror --parallel=2 -v -x '^directory' -x '^another_directory'
FXP (transferring directly FROM one remote FTP server TO another remote FTP server) ∞
get [-E] [-a] [-c] [-O base] rfile [-o lfile] ... Retrieve the remote file rfile and store it as the local file lfile. If -o is omitted, the file is stored to local file named as base name of rfile. You can get multiple files by specifying multiple instances of rfile [and -o lfile]. Does not expand wildcards, use mget for that. -c continue, reget -E delete remote files after successful transfer -a use ascii mode (binary is the default) -O <base> specifies base directory or URL where files should be placed Examples: get README get README -o debian.README get README README.mirrors get README -o debian.README README.mirrors -o debian.mirrors get README -o ftp://some.host.org/debian.README get README -o ftp://some.host.org/debian-dir/ (end slash is important)
The following example will show you a simple way using the LFTP program on a remote client, making the a FTP server to send files to another FTP server - "FXP".
Write the following into a file (e.g. config.file
), and don't forget to remove the [
and ]
.
set ftp:ssl-force true set ftp:use-fxp true open -p [Source Port] [Source Address] user [Source Username] [Source Password] mirror [Source Directory] ftp://[Destination Username]:[Destination Password]@[Destination Address]:[Destination Port]/[Destination Directory]
And execute the file:
\lftp -f config.file
Multiple Commands ∞
It's done just like xterm.
lftp -c " open -u $user,$pass $server ;\ mirror --reverse --continue --delete --parallel=2 ;\ "
This automatically exits from the server upon completion.
-
2010-04-29 -- In this example, no file transfer summary is printed; I don't know how to solve that.
Find content in a file ∞
glob -f more * | grep v1.gif
-
2006-10-06 -- But it won't tell you what file it's in. =/
Leeching a website ∞
user='USERNAME' pass='PASSWORD' site='WEBSITE' \mkdir -p ./LEECH \lftp \ -u "$user","$pass" \ -e "\ set ssl:verify-certificate false && \ mirror --continue --parallel=3 --verbose . ./LEECH/ \ "\ "$site"
Troubleshooting ∞
"Host key verification failed" ∞
2018-02-24 - Tested and works
When you do:
\lftp sftp://username@example.com
.. if you get:
Password: cd: Fatal error: Host key verification failed
.. some combination of the following should work.
--
First, double and triple-check the settings on your web host.
- Check the username.
- Check that FTP is enabled for that user.
- Check that FTP is enabled for that website.
-
Re-try after you simplify your command.
Step one, SSH in:
\ssh USERNAME@example.com
It'll talk to you a bunch.. say yes until it asks you for the website's ftp password. At that point, just control-c
.
Step two:
\touch "$HOME/.ssh/known_hosts" \ssh-keygen -R example.com
"Fatal error: Certificate verification: certificate common name doesn't match requested host name" ∞
edit ~/.config/lftp/rc
and add:
set ssl:check-hostname no
I don't know the consequences of this.
2010-02-02 ∞
-
$HOME/.lftp/transfer_log
keeps records, including the username and password, what THE FUCK..
2006-12-21 ∞
-
I hate stupid error messages like this.. why not simply display the damned error and then the help text?
> mirror --help mirror: unrecognized option `--help' Try `help mirror' for more information.
Last updated 2022-06-05 at 18:51:25
I don't think I'll bother with usage notes. Lots of great examples are in my shell-random repository.
Ported, including my usage notes. Past-self, you coward you.
LFTP is rarely used; I switched to [[rsync]] over [[SSH]]. Flagging zombie.
added website leeching notes