(on Wikipedia)
http://lftp.yar.ru/ [ 1 ]

An FTP-like program which can also Mirror.

I use this to synchronize files over SFTP (SSH).

---

• https://github.com/lavv17/lftp

—

• 2020-03-25 – Some time ago I switched to using rsync over SSH.
• 2020-03-25 – 4.8.4-2 on Debian 10.1.0-amd64-xfce-CD-1
• 2020-03-25 – 4.7.4-1 on Debian 9.9.0-i386-xfce-CD-1

• 2016-04-10 – 4.4.9 on Slackware 14.1

  • Libraries used: Readline 5.2, Expat 2.1.0, OpenSSL 1.0.1s 1 Mar 2016, zlib 1.2.8

• 2016-03-26 – 4.4.13 on Lubuntu 14.04.4 LTS

  • Libraries used: Readline 6.3, GnuTLS 2.12.23, zlib 1.2.8

Intro/Notes ∞

Has some spectacular features:

• Mirroring and “reverse mirroring“
• directory listing caching
• colour lists
• find, grep, ls -al
• job control with ^z, fg, etc.
• !ls for local listing. !<command> for whatever, locally.
• To get into an sftp site with LFTP, do lftp sftp://username@host

• lcd <path> to locally cd and stay there for the purposes of file transfers.

LFTP for BitTorrent ∞

First explored on 2013-06-23 with BitTorrent with the Raspberry Pi

• No encryption support.
• No ability to select individual files to download.
• No ability to set individual file priority.

• Cannot specify/modify a list of trackers. I assume it gets something from a .torrent file, if given one.

  • Yes .torrent files and trackers are technically obsolete, but they still have some value.

I didn’t even research multiple torrents. It’s a safe bet that there’s no prioritization between them. Multiple LFTP torrent processes fighting it out does not sound like a good idea.

Usage ∞

There are scripts scattered throughout https://github.com/spiralofhope/shell-random/tree/master.

You would be better off getting the whole repository, then using its findinall() for \lftp (including that backslash).

Mirroring (download remote files TO your local machine) ∞

• set ftp:list-options -a – Also download files which start with a dot.
• --parallel=2 – Download two files in parallel. I like the idea of flooding my connection to help the overall download speed stay high.
• -v – verbose. Shouldn’t this be a default? Sigh..

• -e – delete. Replicate deletions.

set ftp:list-options -a
mirror --parallel=2 -v

Reverse-mirroring (push local files TO the remote FTP site) ∞

• set ftp:list-options -a – Also download files which start with a dot.
• --parallel=2 – Download two files in parallel. I like the idea of flooding my connection to help the overall download speed stay high.
• -v – verbose. Shouldn’t this be a default? Sigh..
• -R – reverse. Sends files to the remote location.

• -e – delete. Replicate deletions.

set ftp:list-options -a
mirror --parallel=2 -Rv

issues

• 2008-06-20 — I’ve had the FTP connection hang repeatedly and fail on all retries.

• 2008-06-20 — I’ve tried re-sending things, and saw it remove and re-send a significant number of files.

Active Mode ∞

• 2006-07-20 — I don’t know why I can’t just switch between modes while connected, or why I can’t use a switch on the commandline when running LFTP. Presumably because humans didn’t make this program:

I have to edit $HOME/.lftp/rc, which doesn’t exist so I don’t know what to put in it.

The documentation says a bunch of shit which doesn’t mean anything to me. There is no example.

    ftp:passive-mode (boolean)
         sets passive ftp mode. This can be useful  if  you  are
         behind  a  firewall  or  a dumb masquerading router. In
         passive mode lftp uses PASV command, not the PORT  com-
         mand which is used in active mode. In passive mode lftp
         itself makes the data  connection  to  the  server;  in
         active mode the server connects to lftp for data trans-
         fer. Passive mode is the default.

I think set ftp:passive-mode false does do something, but Googling gives me better documentation:

set passive-mode/ftp.site1.com yes
set passive-mode/ftp.site2.org no
set passive-mode/*.site3.net yes

2007-07-09 — This was tested and works. If $HOME/.lftp/rc doesn’t exist, make it.

Regular expressions ∞

(Regular expressions)

LFTP does it like grep, so you’re out of luck if you want to do it in an easy way.

exclude:

-x '^certs' -x '^store'

Like so:

set ftp:list-options -a
mirror --parallel=2 -v -x '^directory' -x '^another_directory'

FXP (transferring directly FROM one remote FTP server TO another remote FTP server) ∞

get [-E] [-a] [-c] [-O base] rfile [-o lfile] ...

Retrieve the remote file rfile and store it as the local file lfile. If -o is omitted, the file is
stored to local file named as base name of rfile. You can get multiple files by specifying multiple
instances of rfile [and -o lfile]. Does not expand wildcards, use mget for that.
-c continue, reget
-E delete remote files after successful transfer
-a use ascii mode (binary is the default)
-O <base> specifies base directory or URL where files should be placed

Examples:
get README
get README -o debian.README
get README README.mirrors
get README -o debian.README README.mirrors -o debian.mirrors
get README -o ftp://some.host.org/debian.README
get README -o ftp://some.host.org/debian-dir/ (end slash is important)

props to https://web.archive.org/web/20100227095930/http://www.linuxforums.org:80/forum/linux-applications/64567-fxp-lftp.html

The following example will show you a simple way using the LFTP program on a remote client, making the a FTP server to send files to another FTP server – “FXP”.

Write the following into a file (e.g. config.file), and don’t forget to remove the [ and ].

set ftp:ssl-force true
set ftp:use-fxp true

open -p [Source Port] [Source Address]
user [Source Username] [Source Password]
mirror [Source Directory] ftp://[Destination Username]:[Destination Password]@[Destination Address]:[Destination Port]/[Destination Directory]

And execute the file:

\lftp  -f config.file

Multiple Commands ∞

It’s done just like xterm.

lftp -c "
  open -u $user,$pass $server ;\
  mirror --reverse --continue --delete --parallel=2 ;\
  "

This automatically exits from the server upon completion.

• 2010-04-29 — In this example, no file transfer summary is printed; I don’t know how to solve that.

Find content in a file ∞

glob -f more * | grep v1.gif

• 2006-10-06 — But it won’t tell you what file it’s in. =/

Leeching a website ∞

user='USERNAME'
pass='PASSWORD'
site='WEBSITE'

\mkdir  -p  ./LEECH

\lftp \
  -u "$user","$pass" \
  -e "\
    set ssl:verify-certificate false && \
    mirror --continue --parallel=3 --verbose . ./LEECH/ \
  "\
  "$site"

Troubleshooting ∞

“Host key verification failed” ∞

2018-02-24 – Tested and works

When you do:

\lftp  sftp://username@example.com

.. if you get:

Password: 
cd: Fatal error: Host key verification failed

.. some combination of the following should work.

—

First, double and triple-check the settings on your web host.

• Check the username.
• Check that FTP is enabled for that user.
• Check that FTP is enabled for that website.

• Re-try after you simplify your command.

Step one, SSH in:

\ssh  USERNAME@example.com

It’ll talk to you a bunch.. say yes until it asks you for the website’s ftp password. At that point, just control-c.

Step two:

\touch  "$HOME/.ssh/known_hosts"
\ssh-keygen  -R example.com

“Fatal error: Certificate verification: certificate common name doesn’t match requested host name” ∞

edit ~/.config/lftp/rc and add:

set ssl:check-hostname no

I don’t know the consequences of this.

2010-02-02 ∞

• $HOME/.lftp/transfer_log keeps records, including the username and password, what THE FUCK..

2006-12-21 ∞

• I hate stupid error messages like this.. why not simply display the damned error and then the help text?

> mirror --help
mirror: unrecognized option `--help'
Try `help mirror' for more information.

Footnotes

1. was https://web.archive.org/web/20210320235338/http://lftp.tech/[ ↩ ]