Golang FTP Server library

This library allows to easily build a simple and fully-featured FTP server using afero as the backend filesystem.

If you're interested in a fully featured FTP server, you should use sftpgo (fully featured SFTP/FTP server) or ftpserver (basic FTP server).

Current status of the project

Features

• Uploading and downloading files
• Directory listing (LIST + MLST)
• File and directory deletion and renaming
• TLS support (AUTH + PROT)
• File download/upload resume support (REST)
• Passive socket connections (PASV and EPSV commands)
• Active socket connections (PORT and EPRT commands)
• IPv6 support (EPSV + EPRT)
• Small memory footprint
• Clean code: No sleep, no panic, no global sync (only around control/transfer connection per client)
• Uses only the standard library except for:
  • afero for generic file systems handling
• Uses standard library log/slog for structured logging
• Supported extensions:
  • AUTH - Control session protection
  • AUTH TLS - TLS session
  • PROT - Transfer protection
  • EPRT/EPSV - IPv6 support
  • MDTM - File Modification Time
  • SIZE - Size of a file
  • REST - Restart of interrupted transfer
  • MLST - Simple file listing for machine processing
  • MLSD - Directory listing for machine processing
  • HASH - Hashing of files
  • AVBL - Available space
  • COMB - Combine files
  • MODE Z - Transfer compression (deflate)

Quick test

The easiest way to test this library is to use ftpserver.

The driver

The simplest way to get a good understanding of how the driver shall be implemented is to look at the tests driver.

The base API

The API is directly based on afero.

// MainDriver handles the authentication and ClientHandlingDriver selection
type MainDriver interface {
	// GetSettings returns some general settings around the server setup
	GetSettings() (*Settings, error)

	// ClientConnected is called to send the very first welcome message
	ClientConnected(cc ClientContext) (string, error)

	// ClientDisconnected is called when the user disconnects, even if he never authenticated
	ClientDisconnected(cc ClientContext)

	// AuthUser authenticates the user and selects an handling driver
	AuthUser(cc ClientContext, user, pass string) (ClientDriver, error)

	// GetTLSConfig returns a TLS Certificate to use
	// The certificate could frequently change if we use something like "let's encrypt"
	GetTLSConfig() (*tls.Config, error)
}


// ClientDriver is the base FS implementation that allows to manipulate files
type ClientDriver interface {
	afero.Fs
}

// ClientContext is implemented on the server side to provide some access to few data around the client
type ClientContext interface {
	// Path provides the path of the current connection
	Path() string

	// SetDebug activates the debugging of this connection commands
	SetDebug(debug bool)

	// Debug returns the current debugging status of this connection commands
	Debug() bool

	// Client's ID on the server
	ID() uint32

	// Client's address
	RemoteAddr() net.Addr

	// Servers's address
	LocalAddr() net.Addr

	// Client's version can be empty
	GetClientVersion() string

	// Close closes the connection and disconnects the client.
	Close() error

	// HasTLSForControl returns true if the control connection is over TLS
	HasTLSForControl() bool

	// HasTLSForTransfers returns true if the transfer connection is over TLS
	HasTLSForTransfers() bool

	// GetLastCommand returns the last received command
	GetLastCommand() string

	// GetLastDataChannel returns the last data channel mode
	GetLastDataChannel() DataChannel
}

// Settings define all the server settings
type Settings struct {
	Listener                 net.Listener     // (Optional) To provide an already initialized listener
	ListenAddr               string           // Listening address
	PublicHost               string           // Public IP to expose (only an IP address is accepted at this stage)
	PublicIPResolver         PublicIPResolver // (Optional) To fetch a public IP lookup
	PassiveTransferPortRange PasvPortGetter   // (Optional) Port Range for data connections. Random if not specified
	PassiveTransferPortMultiplexing bool      // Allow different client IPs to share passive listener ports
	ActiveTransferPortNon20  bool             // Do not impose the port 20 for active data transfer (#88, RFC 1579)
	IdleTimeout              int              // Maximum inactivity time before disconnecting (#58)
	ConnectionTimeout        int              // Maximum time to establish passive or active transfer connections
	DisableMLSD              bool             // Disable MLSD support
	DisableMLST              bool             // Disable MLST support
	DisableMFMT              bool             // Disable MFMT support (modify file mtime)
	Banner                   string           // Banner to use in server status response
	TLSRequired              TLSRequirement   // defines the TLS mode
	DisableLISTArgs          bool             // Disable ls like options (-a,-la etc.) for directory listing
	DisableSite              bool             // Disable SITE command
	DisableActiveMode        bool             // Disable Active FTP
	EnableHASH               bool             // Enable support for calculating hash value of files
	DisableSTAT              bool             // Disable Server STATUS, STAT on files and directories will still work
	DisableSYST              bool             // Disable SYST
	EnableCOMB               bool             // Enable COMB support
	DefaultTransferType      TransferType     // Transfer type to use if the client don't send the TYPE command
	// ActiveConnectionsCheck defines the security requirements for active connections
	ActiveConnectionsCheck DataConnectionRequirement
	// PasvConnectionsCheck defines the security requirements for passive connections
	PasvConnectionsCheck DataConnectionRequirement
}

Extensions

There are a few extensions to the base afero APIs so that you can perform some operations that aren't offered by afero.

Pre-allocate some space

// ClientDriverExtensionAllocate is an extension to support the "ALLO" - file allocation - command
type ClientDriverExtensionAllocate interface {

	// AllocateSpace reserves the space necessary to upload files
	AllocateSpace(size int) error
}

Get available space

// ClientDriverExtensionAvailableSpace is an extension to implement to support
// the AVBL ftp command
type ClientDriverExtensionAvailableSpace interface {
	GetAvailableSpace(dirName string) (int64, error)
}

Create symbolic link

// ClientDriverExtensionSymlink is an extension to support the "SITE SYMLINK" - symbolic link creation - command
type ClientDriverExtensionSymlink interface {

	// Symlink creates a symlink
	Symlink(oldname, newname string) error

	// SymlinkIfPossible allows to get the source of a symlink (but we don't need for now)
	// ReadlinkIfPossible(name string) (string, error)
}

Compute file hash

// ClientDriverExtensionHasher is an extension to implement if you want to handle file digests
// yourself. You have to set EnableHASH to true for this extension to be called
type ClientDriverExtensionHasher interface {
	ComputeHash(name string, algo HASHAlgo, startOffset, endOffset int64) (string, error)
}

Detect transfer errors and interruptions

FTP has no length header for STOR uploads, so the server can't always tell whether an upload that "ended" did so because the client sent every byte or because it gave up. To get notified when the server does detect an interruption (ABOR, dropped TCP connection, I/O error), implement the FileTransferError interface on the value you return from your afero.Fs OpenFile/Create (or from ClientDriverExtentionFileTransfer.GetHandle):

type monitoredFile struct {
    afero.File
    transferErr error
}

// TransferError is called before Close whenever the transfer did not
// complete normally (ABOR, broken connection, copy error, ...).
func (f *monitoredFile) TransferError(err error) {
    f.transferErr = err
}

func (f *monitoredFile) Close() error {
    err := f.File.Close()
    if f.transferErr != nil {
        // upload was interrupted - run your "discard partial file" logic
    } else {
        // upload completed - run your "publish file" logic
    }
    return err
}

TransferError is always invoked before Close. Both methods are called on the same FileTransfer instance, so a simple flag is enough to distinguish the two cases. A client that closes the data connection mid-upload without sending ABOR is indistinguishable from a clean completion - this is a limitation of the FTP protocol itself, not of this library.

History of the project

I wanted to make a system which would accept files through FTP and redirect them to something else. Go seemed like the obvious choice and it seemed there was a lot of libraries available but it turns out none of them were in a useable state.

• micahhausler/go-ftp is a minimalistic implementation
• shenfeng/ftpd.go is very basic and 4 years old.
• yob/graval is 3 years old and “experimental”.
• goftp/server seemed OK but I couldn't use it on both Filezilla and the MacOs ftp client.
• andrewarrow/paradise_ftp - Was the only one of the list I could test right away. This is the project I forked from.