Copyright (C) 2001, 2002, 2003, 2004, 2007, 2008 Wojciech Polak and Sergey Poznyakoff. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover texts being "A GNU Manual", and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled "GNU Free Documentation License". (a) The FSF's Back-Cover Text is: "You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development." 1 GNU Anubis ************ This edition of the `GNU Anubis Manual', last updated 20 December 2008, documents GNU Anubis Version 4.1.1. 2 Overview ********** GNU Anubis is an SMTP message submission daemon. Its purpose is to receive the outgoing message, perform some manipulations over its contents, and to forward the altered message to the mail transport agent. The usual mail sending scheme looks as follows: a user composes his message using "mail user agent" ("MUA" for short). Once the message is composed, the user sends it. When the MUA receives the send command it connects to the "mail transport agent" ("MTA" for short) and passes it the message for delivery. The figure below illustrates this interaction: +-------+ +-------+ | MUA | ---[outmsg]---> | MTA | ... [outmsg] +-------+ +-------+ | | V +--------------+ | Recipient's | | Mailbox | +--------------+ As shown in this figure, the outgoing message ("outmsg"), reaches the recipient's mailbox unaltered. However, there are situations where it may be necessary to modify the outgoing message before it reaches MTA. As the simplest example, the user might wish to sign outgoing messages with his PGP key, but his MUA does not support this operation or supports it unconditionally. In such cases, installing GNU Anubis between the MUA and MTA allows the user to perform any additional processing on the sent message. The figure below illustrates this concept: +-------+ +--------+ +-------+ | MUA | ---[outmsg]---> | Anubis | ---[modmsg]---> | MTA | +-------+ +--------+ +-------+ | [modmsg] . . V +--------------+ | Recipient's | | Mailbox | +--------------+ The outgoing message is modified by GNU Anubis, and it is the resulting message ("modmsg") that reaches the MTA. GNU Anubis is able to perform a wide set of operations on messages, such as modifying headers or body, encrypting or signing messages with GPG (GNU Privacy Guard) keys, installing secure tunnels to MTA using TLS/SSL encryption, tunneling messages through SOCKS proxies, etc. When the set of built-in operations is not enough, the user can define his own operations using Guile, a "GNU's Ubiquitous Intelligent Language for Extensions". Apart from configurable operations, GNU Anubis always performs "SMTP session normalization", a process that ensures that the SMTP stream coming out of Anubis complies with the RFC 2821, even if the incoming stream does not. In particular, Anubis removes any extra whitespace appearing between `MAIL FROM:' or `SMTP TO' command and its argument. Message processing is controlled by system-wide and per-user configuration files written in a flexible and easy to use command scripting language, specially designed for this purpose. 3 Glossary of Frequently Used Terms *********************************** "Authentication" A process whereby Anubis determines the authenticity of the connecting party, its user name and configuration settings. "Protocol" Any standard for the exchange of information. A protocol defines the specific wording and control flow for communications between two or more programs, devices, or systems. "SMTP" Simple Mail Transport Protocol is a common mechanism for exchanging mail across a network. This protocol is described in the RFC 821 document. "Daemon" We use a term "daemon" to define a process that runs in the background, doing automated processing. "Server" A server provides information or other services for its clients. Most network protocols are client-server based. This term usually refers to an entire machine, but it can refer (and we're doing that) also to the particular program or process, on that machine, that provides the service. "Proxy" We use a term "proxy" to define a program, which goes between the MUA and the MTA (it makes a tunnel). It can be used as a gateway to the outside world, while using a firewall. In this case the host under the firewall sends data to the proxy server, which in turn forwards it to the real server outside, receives the response, and passes it back to the internal host. "Guile" GNU's Ubiquitous Intelligent Language for Extensions. It provides a Scheme interpreter conforming to the R4RS language specification. GNU Anubis uses Guile as its extension language. For more information about Guile, *note Overview: (guile)Top. "GPG" GNU Privacy Guard, a tool compatible with the Pretty Good Privacy. 4 Authentication **************** When GNU Anubis accepts an incoming connection, it first has to identify the remote party, i.e. determine whether it has the right to use Anubis resources and, if so, what configuration settings should be used during the session. We call this process "authentication". The exact method of authentication depends on Anubis "operation mode". Currently there are two modes: transparent This is the default mode. It is compatible with versions of GNU Anubis up to 3.6.2. In this mode, Anubis relies on AUTH service (`identd') to authenticate users. auth This mode uses SMTP AUTH mechanism to authenticate incoming connections. *Note Pixie-Dixie::, this is the first draft description of this mode. Both modes have their advantages and deficiencies, which you have to weigh carefully before choosing which one to use. These are discussed below: Transparent (`traditional') mode. ================================= Deficiencies: 1. The user must have `identd' installed on his machine. 2. Each user must have a system account on the machine where GNU Anubis runs (though the system administrator may relax this limitation using user name translation, *note TRANSLATION Section::). Advantages: 1. Relative simplicity. You don't have to create your users database. 2. Authentication is performed immediately after the connection. Auth mode. ========== Deficiencies: 1. You have to maintain your users database 2. User's MUA must be able to perform ESMTP AUTH.(1) Advantages: 1. Better reliability. 2. Users do not have to run `identd' on their machines. 3. Users are not required to have accounts on the machine where Anubis runs. 4. Users can remotely modify their configuration files. ---------- Footnotes ---------- (1) It is not a serious restriction, however. The user may install Anubis on his machine for the sole purpose of SMTP authentication, as Pixie-Dixie suggests. 4.1 User Database ================= "User Database" is a place where GNU Anubis uses keeps "user credentials", i.e. data used to authenticate and authorize users. The exact way of storing these data does not matter here, it will be addressed further in this manual. In this section we treat user database as an abstraction layer. The user database consists of "records". Each record keeps information about a particular "user". A record consists of four "fields". A field may contain some value, or be empty, in which case we say that the field has "null" value. The record fields are: `SMTP AUTHID' SMTP authentication ID of the user. `AUTH PASSWORD' SMTP password. `ACCOUNT' System user name to be used. `CONFIG' Path to the configuration file. The first two fields are mandatory and must always have non-null values. No two records in the database may have the same value of `SMTP AUTHID' field. When `anubis' is trying to authenticate a user, it first looks up in the database a record with the value of `SMTP AUTHID' field matching AUTHID given by the user. If no such entry is found, authentication fails. Otherwise, `anubis' goes on and compares the password supplied by the user with that from `AUTH PASSWORD' column. If these match, authentication succeeds and `anubis' passes to authorization state. In this state, it first determines the user ID (UID) to switch to. If the `ACCOUNT' field is not null, its value is used as a login name of the system account to use. If it is null, `anubis' switches to the privilege level of a "default not privileged user", specified by `user-notprivileged' statement in the global configuration file (*note user-notprivileged: Security Settings.). The final step is to parse "user configuration file". If `CONFIG' field is not null, its value is used as absolute path to the configuration file. Otherwise, `anubis' searches for file `~/.anubisrc' (where `~' denotes home directory for the system account obtained on the previous step) and if such a file exists, loads it. 4.2 Database URL ================ Anubis database is identified by its "URL", or "Universal Resource Locator". A `URL' consists of following elements (square brackets enclose optional elements): PROTO://[[USER[:PASSWORD]@]HOST]/PATH[PARAMS] The detailed description of each `URL' part follows: PROTO Specifies a database "protocol". The protocol describes how the database is to be accessed. In a way, it may be regarded as specifying the database "type". Currently, GNU Anubis supports following database protocols: `text' A plain text file, containing users' credentials. `gdbm' GDBM database `mysql' MySQL database `pgsql' PostgreSQL database `postgres' Alias for `pgsql'. These protocols are described in detail below. USER User name necessary to access the database. PASSWORD User password necessary to access the database. HOST Domain name or IP address of a machine running the database. PATH A "path" to the database. The exact meaning of this element depends on the database protocol. It is described in detail when discussing particular database protocols. PARAMS A list of protocol-dependent parameters. Each parameter is of the form `KEYWORD=NAME', parameters are separated by semicolons. 4.2.1 Plain text databases -------------------------- This is the simplest database possible. It is kept in a plain text file. Each line in this file represents a single "record", empty lines and lines beginning with `#' ("comments") sign are ignored. Records consist of "fields", each field being a sequence of characters. Fields are separated by colons (`:', ASCII 58). If `:' character occurs in a field, it is preceeded by a single backslash character (`\\', ASCII 92). A record must contain at least two fields. 1. SMTP `AUTHID'. 2. SMTP password. 3. Account name. 4. Path to user configuration file. URL syntax .......... The URL syntax for this type of databases is quite simple: text:PATH where PATH specifies absolute file name of the database file. 4.2.2 Databases in GDBM format ------------------------------ The protocol value `gdbm' specifies a "GDBM database". For the detailed description of GDBM system *note Introduction: (gdbm)Top. URL syntax for GDBM databases is: gdbm:PATH where PATH specifies absolute file name of the database file. 4.2.3 MySQL and PostgreSQL -------------------------- This is the most flexible database format. GNU Anubis 4.1.1 supports MySQL(1) and PostgreSQL(2) interfaces. No matter which of them you use, the implementation details are hidden behind a single consistent Anubis interface. GNU Anubis supposes that all user data are kept in a single database table. This table must have at least four columns for storing SMTP `AUTHID', SMTP password, system account name and path to user configuration file. Among those, only the last two may have NULL values. There is no restriction on the name of the database or the authentication table, nor on its column names. This information may be specified in URL as discussed below. URL syntax .......... PROTO://[[USER[:PASSWORD]@HOST/DBNAME[PARAMS] PROTO describes the exact database type to use. Use `mysql' for MySQL databases and `pgsql' or `postgres' for PostgreSQL databases. Optional USER and PASSWORD specify authentication credentials used to access the database. HOST sets domain name or IP address of the machine running the database. It may be omitted if the database resides on `localhost'. The database name is specified by DBNAME element. Finally, further details needed for connecting to the database may be given by URL parameters. All of them have reasonable default values, so you'll have to specify only those parameters that does not match the default values. Known parameters are: PORT=NUMBER Specifies the port number to be used when connecting to the database. If it is not specified, the behavior depends on the value of SOCKET parameter: if SOCKET is not present, the program will use the default port number for the given protocol (i.e. 3306 for `mysql' and 5432 for `pgsql'. SOCKET=STRING Specifies UNIX name of the socket to connect to. This parameter cannot be used together with PORT (see above). BUFSIZE=NUMBER Sets the length of the buffer used to create SQL queries. Default is 1024 bytes. TABLE=STRING Specifies the name of database table keeping where the authentication data are stored. Default is `users'. AUTHID=STRING Specifies the name of a column in TABLE which holds `AUTHID' value. Default is `authid'. PASSWD=STRING Specifies the name of a column in TABLE which holds user password. Default is `passwd'. ACCOUNT=STRING Specifies the name of a column in TABLE which holds the name of system account to be used for this `AUTHID'. Default is `account'. RCFILE=STRING Specifies the name of a column in TABLE which holds path to the user's configuration file. Default is `rcfile'. ---------- Footnotes ---------- (1) See `http://www.mysql.com'. (2) See `http://www.postgres.org'. 4.3 Managing the Database ========================= Managing the user database is a complex task, which looks differently from administrator's and user's point of view. The administrator have full rights on the database, it can add new records and delete or modify existing ones. A user, of course, does not have such ample rights. The only thing he is able to do is to maintain his own record in the database, provided that he already has one. If he does not, he should contact the system administrator and arrange for the creation of his record. 4.3.1 Administrators -------------------- All administrative tasks are done using `anubisadm' command -- a multipurpose tool for Anubis administrator. The command usage syntax is: anubisadm COMMAND [OPTIONS] DATABASE-URL where COMMAND specifies the operation to be performed on the database, OPTIONS give additional operation-specific parameters, and DATABASE-URL specifies the database to operate upon. All administrative tasks can be subdivided into the following five categories: * Creating the Database * Listing Database Records * Adding New Records * Removing Existing Records * Modifying Existing Records These operations are described in detail in the following subsections. 4.3.1.1 Creating the Database ............................. To create a database use `anubisadm --create' (or `anubisadm -c') command. `Anubisadm' will read database entries from the standard input and write them to the database. The standard input is supposed to be formatted as `text' database (*note text::). Thus to create a GDBM database from plain text file `userlist', use the following command anubisadm --create gdbm:/etc/anubis.db < userlist Similarly, to create an initially empty database, type anubisadm --create gdbm:/etc/anubis.db < /dev/null _Notice_, that if you use SQL database format, `--create' command does not imply creating the database structure! So, before running anubisadm --create mysql://localhost/dbname < userlist make sure you create the underlying database structure (including granting privileges to the `anubis' user), via the usual procedure. Please refer to corresponding database manual for the detailed instructions on this. It is sometimes necessary to convert the existing user database from one format (protocol) to another. For example, suppose you have been running GDBM database (`text:/etc/anubis.db') for some time, but now it has grown considerably and you decided to switch to PostgreSQL database to improve performance. To do so, first create the database using postgres utilities. Then run anubisadm --list text:/etc/anubis.db | \ anubisadm --create pgsql://localhost/dbname That's all there is to it! 4.3.1.2 Listing Database Records ................................ The command `--list' (or `-l') lists the existing database. When run without additional options, it will display all records from the database, e.g.: anubisadm --list gdbm:/etc/anubis.db Among its other uses, such invocation is handy for converting user database to another format (*note Create::). If you wish to list only a particular record, specify the `AUTHID' using `--authid' (`-i') option. For example, to list record of the user with `AUTHID' `test', type: example$ anubisadm --list --authid test gdbm:/etc/anubis.db 4.3.1.3 Adding New Records .......................... To add a new record use command `--add' (`-a'). Additional data are specified via the following options: `-i STRING' `--authid=STRING' Specify the user `SMTP AUTHID'. `-p STRING' `--password=STRING' Specify user password password. `-u STRING' `--user=STRING' Specify system user name corresponding to the given `AUTHID'. `-f STRING' `--rcfile=STRING' Specify configuration file to be used for this user. For example, the following command adds a record with `SMTP AUTHID' `test', password `guessme' and maps it to the system account `gray': anubisadm --add --authid test --password guessme \ --user gray gdbm:/etc/anubis.db 4.3.1.4 Removing Existing Records ................................. Removing a record is quite straightforward: use `--remove' (`-r') command and specify `AUTHID' using `--authid' option. For example, to remove the record created in the previous subsection, run: anubisadm --remove --authid test gdbm:/etc/anubis.db 4.3.1.5 Modifying Existing Records .................................. To modify an existing record use command `--modify' (`-m'). The record is identified via `--authid' option. The fields to be changed are given with the following options: `-p STRING' `--password=STRING' Specify user password password. `-u STRING' `--user=STRING' Specify system user name corresponding to the given `AUTHID'. `-f STRING' `--rcfile=STRING' Specify configuration file to be used for this user. For example, the following command sets new configuration file name for the user `smith': anubisadm --authid smith \ --rcfile=/var/spool/anubis/common gdbm:/etc/anubis.db 4.3.1.6 Summary of All Administrative Commands .............................................. * Usage anubisadm COMMAND [OPTIONS] DATABASE-URL * Commands: `-c' `--create' Create the database. `-l' `--list' List the contents of an existing database. `-a' `--add' Add a new record. `-m' `--modify' Modify an existing record. `-r' `--remove' Remove an existing record. `--version' Display program version number and exit. `--help' Display short usage summary and exit. * Options: `-i STRING' `--authid=STRING' Specify the authid to operate upon. This option is mandatory for `--add', `--modify' and `--remove' commands. It may also be used with `--list' command. `-p STRING' `--password=STRING' Specify the password for the authid. This option is mandatory for `--add', `--modify' and `--remove' commands. `-u STRING' `--user=STRING' Specify the system user name corresponding to the given authid. It may be used with `--add', `--modify', and `--remove' commands. `-f STRING' `--rcfile=STRING' Specify the rc file to be used for this authid. The option may be used with `--add', `--modify', and `--remove' commands. 4.3.2 Users ----------- Users maintain their database records using `anubisusr' command. Main purpose of this command is to keep the copy of your configuration on GNU Anubis server up to date. . We recommend to invoke `anubisusr' from your `~/.profile', which will make sure that your configuration file is up to date when you log in. (1). Usage ..... anubisusr [OPTIONS] [SMTP-URL] where SMTP-URL is a URL of your GNU Anubis server. Notice that if it lacks user name and password, then `anubisusr' will first try to retrieve them from your `~/.netrc' file (See `netrc(5)' for more info), and if not found it will prompt you to supply them. Options ....... `-m MECH' `--mechanism MECH' Only use SASL mechanism MECH. Use this option several times to set a list of allowed mechanisms. `-v' `--verbose' Verbose output. Multiple options increase the verbosity. Maximum verbosity level is 3. `--version' Display program version number and exit. `--help' Display short usage summary and exit. ---------- Footnotes ---------- (1) Make sure to run `anubisusr' in background, so it does not slow down your normal login sequence 5 Configuration *************** The behavior of GNU Anubis is controlled by two configuration files. The "system configuration file", `/etc/anubisrc', specifies system-wide options that affect all users. This file is usually owned by root. The "user configuration file" specifies what GNU Anubis should do for a particular user. By default it is located in `~/.anubisrc'. This location can be changed in auth mode. To protect your passwords in the configuration files, use the 0600 (u=rw,g=,o=) permissions, otherwise GNU Anubis won't accept them. Lexical Structure ----------------- Both configuration files use simple line-oriented syntax. Each line introduces a single statement. A statement consists of "words", each word being defined as a contiguous sequence of non-whitespace symbols. A word may be composed of alphanumeric characters and any of the following punctuation symbols: `_', `.', `/', `-'. Any arbitrary sequence of characters enclosed in a pair of double quotes is also recognized as a word. Such a sequence is called "quoted string". Quoted strings follow the same syntax rules as in C language. A backslash character `\' alters the meaning of the character following it. This special construct is called "escape sequence". When processing an escape sequence, Anubis removes it from the string and replaces it with a single character as described in the following table: \a Audible bell character (ASCII 7) \b Backspace (ASCII 8)(1) \e Escape character (ASCII 27) \f Form feed (ASCII 12) \n Newline (ASCII 10) \r Carriage return (ASCII 13) \t Horizontal tab (ASCII 9) \d (where `d' represents a single decimal digit) Reproduced verbatim. A backslash followed by any character not listed above is replaced by the character alone. This can be used for inserting `"' character within a string, as in the example below: "This string contains \"quoted string\"." Similarly, backslash followed by a newline is replaced by the newline itself. Thus, the following two strings are equivalent: "This string is split\nover two lines" "This string is split\ over two lines" The familiar shell "here document" syntax may be used to produce a word containing several lines of text. The syntax is: <<[-]delimiter text delimiter If "here document" starts with `<<-', then all leading tab characters are stripped from input lines and the line containing "delimiter". This allows to indent here-document in a natural fashion. To summarize all the above, let's consider the example: first-word "second word" <<-EOT Third word containing several lines of text EOT This line contains three words: `first-word', `second word' and the third one composed of the three lines between the `EOT' markers. If a statement is very long, it may be split among several lines of text. To do so, precede the newline characters with a backslash `\', e.g.: a very long statement\ occupying several lines\ of text A `#' in a line starts a "comment". It and the rest of the line are ignored. Comments may appear on any of the lines in the configuration file, except on a commands and within a "here-document" construction. A line containing just a comment (with perhaps spaces before it) is effectively blank, and is ignored. For example: # This is a comment if header[Subject] :re "No.*" # This is also a comment guile-process action-name This # is not a comment!!! fi Logical Structure ----------------- The statements within a configuration file are grouped into "sections". Each section has its name. A section begins with one of the following constructs: BEGIN NAME ---BEGIN NAME--- and ends with one of the following constructs: END ---END--- Notice, that both `BEGIN' and `END' must be uppercase. When using the second form, any amount of whitespace is allowed between the three dashes and the word. The sections cannot be nested. There are five predefined sections, whose names are uppercase. The user may define his own sections, which may then be referred to from the `RULE' section as subroutines (*note Call Action::). The predefined section names are: "AUTH" Controls authentication mechanisms. "CONTROL" This section specifies the basic GNU Anubis behavior. Its presence is required in the system configuration file. It may be used in the user configuration file to override the system-wide settings. "TRANSLATION" This section specifies a translation map for remapping remote or local users. It may be used only in the system-wide configuration file. "GUILE" Contains the settings of the Guile interpreter. The section is allowed in both configuration files. "RULE" Defines the rules that are used to alter the contents of the messages (conditional and unconditional rules). ---------- Footnotes ---------- (1) This escape is not supported by Anubis 4.0 5.1 AUTH Section ================ `AUTH' session controls various aspects of authentication mode. -- Option: smtp-greeting-message TEXT Configures the greeting message issued by GNU Anubis upon accepting the connection. -- Option: smtp-help-message HELP-TEXT Sets the test of the message issued by Anubis in response to SMTP `HELP' command. HELP-TEXT is a list of strings. Each string from the list will be displayed at a separate response line. -- Option: sasl-password-db URL Sets the user database URL (*note User Database::). -- Option: sasl-allowed-mech MECH-LIST Defines the list of allowed authentication methods. -- Option: sasl-service NAME Set SASL "service name". It is used, among others, with GSSAPI authentication method. Default is `anubis'. -- Option: sasl-hostname NAME Set SASL hostname. By default, the server determines it automatically. If, however, it makes a wrong guess, you can fix it using this directive. -- Option: sasl-realm NAME Set SASL realm. By default, the domain part of the current hostname is used as SASL realm. 5.2 CONTROL Section =================== The `CONTROL' section specifies the basic GNU Anubis behavior. Specified in the system configuration file, it applies to all users on the machine, but each user can specify its own `CONTROL' section, to customize own settings. Of course, not all options can be set or changed by user. Some options can only be set in the system configuration file, and some only in user configuration file. By default, options specified in user configuration file have a *higher* priority that those specified in system configuration file. All option names are case insensitive, so you can use for instance: `bind' or `BIND' or `BiNd', and so on. 5.2.1 Basic Settings -------------------- _(This message will disappear, once this node revised.)_ -- Option: bind [HOST:]PORT Specify the TCP port on which GNU Anubis listens for connections. The default HOST value is `INADDR_ANY', which means that anyone can connect to GNU Anubis. The default PORT number is 24 (private mail system). This option is available only in the system configuration file. If you would like, for instance, to bind GNU Anubis to port 25 (SMTP) and limit its clients only to those from `localhost', then set the following in your system configuration file: bind localhost:25 -- Option: remote-mta HOST[:PORT] Specify a remote SMTP host name or IP address, which GNU Anubis will connect and forward mail to (after processing). The default PORT number is 25. This option is available in both configuration files. -- Option: local-mta FILE-NAME [ARGS] Execute a local SMTP server, which works on standard input and output (inetd-type program). This option excludes the `remote-mta' keyword (or `--remote-mta' command line option). For example: local-mta /usr/sbin/sendmail -bs -- Option: mode MODE-NAME Selects Anubis operation mode. Allowed values for MODE-NAME are: transparent auth *Note Authentication::, for the detailed discussion of GNU Anubis operation modes. -- Option: read-entire-body YES-OR-NO When processing a multi-part message using an external filter (*note External Processor::) Anubis normally feeds to it only the first part. The rest of the message is copied verbatim. To alter this behavior so that your external program sees the entire message body, set `read-entire-body yes' in your control section. -- Option: incoming-mail-rule STRING Declares name of processing sections for incoming mail. Default is `INCOMING'. This option is available only for system configuration file. *Note MDA Mode::, for detailed description of incoming mail processing. -- Option: outgoing-mail-rule STRING Declares name of processing sections for outgoing mail. Default is `RULE'. This option is available only for system configuration file. 5.2.2 Output Settings --------------------- -- Option: termlevel LEVEL This is a logging level for `syslogd' or a terminal (if using the `--foreground' command line option). LEVEL can be one of the following: normal Only errors are logged. This is the default level. verbose Produce more diagnostic output. debug Produce debugging output. silent Do not log anything. This command may be used only in system configuration file. -- Option: logfile FILE-NAME This command specifies an additional file, where GNU Anubis can log its information, but only those information available for a client. Only in user configuration file. For example: logfile "anubis.log" This will log to the `~/anubis.log' file in a client's home directory. -- Option: loglevel LEVEL This option specifies an output level for an additional file (`logfile'). It can be used only in user configuration file. LEVEL is one of the following: none fails all -- Option: tracefile YES-OR-NO -- Option: tracefile FILE-NAME This option instructs `anubis' to log the execution of tests and actions from the RULE sections. This is useful for debugging the configuration files. When this option is used in the system-wide configuration file, only its first form is allowed. Using `tracefile yes' enables logging of the actions and tests to the default syslog channel. Using `tracefile no' disables it. When used in the user configuration file, a filename is allowed as an argument to this option. This allows you to explicitly specify to which file the tracing output should go. Otherwise, using `tracefile yes' enables logging to the same file as `logfile' (if possible). -- Option: HANG DELAY _Do not use this option, unless you are developing or debugging Anubis!_ This option instructs a child process to hang for the given number of seconds. Before hanging, the process will issue the following diagnostic message, no matter what the settings of `termlevel' variable were: Child process suspended for DELAY seconds This option is useful for Anubis developers who wish to attach to a child process with debugger. After attaching, set the variable `_anubis_hang' to zero to continue processing. It is useful to add the following statement to your `.gdbinit' file: set variable _anubis_hang=0 5.2.3 Proxy Settings -------------------- -- Option: socks-proxy HOST[:PORT] This option enables tunneling the connections through a SOCKS proxy server, specified as an argument HOST. The PORT default value is 1080, which is a common port number for SOCKS proxies. -- Option: socks-v4 YES-OR-NO This specifies a SOCKS protocol version 4. By default it is turned off, and a default mode is SOCKS protocol version 5. -- Option: socks-auth USERNAME:PASSWORD Specify a user name and a password, if a SOCKS proxy server requires them. A USERNAME and a PASSWORD are separated with a colon (`:'). 5.2.4 ESMTP Authentication Settings ----------------------------------- The following options set authentication credentials for ESMTP authentication. You may use this option, for example, if your MTA requires such an authentication, but your MUA does not support it. -- Option: esmtp-allowed-mech MECH-LIST Defines the list of allowed authentication mechanisms. MECH-LIST is a list of valid authentication mechanism names separated by whitespace. Anubis selects the authentication method using following algorithm: The MTA presents the list of authentication methods it supports. For each element in MECH-LIST, Anubis tests whether it is available in the list presented by MTA. If found, this method is selected. For example, suppose that the MTA supports following mechanisms: PLAIN LOGIN CRAM-MD5 ANONYMOUS and you have following statement in your configuration file esmtp-allowed-mech DIGEST-MD5 CRAM-MD5 LOGIN In this case Anubis will select CRAM-MD5. -- Option: esmtp-require-encryption MECH-LIST This statement declares the list of mechanisms that can be used only over a TLS encrypted channel. By default Anubis uses esmtp-require-encryption LOGIN PLAIN This prevents sending user password over an unencrypted connection. -- Option: esmtp-auth-id AUTHENTICATION-ID Sets authentication ID (user name). -- Option: esmtp-authz-id AUTHORIZATION-ID Sets authorization ID (user name). -- Option: esmtp-password PASSWORD Sets password to be used in authentication. -- Option: esmtp-auth USERNAME:PASSWORD This option sets both authentication and authorization IDs and the password. It is equivalent to esmtp-auth-id USERNAME esmtp-authz-id USERNAME esmtp-password PASSWORD The following options specify authentication credentials for GSSAPI, DIGEST-MD5 and KERBEROS_V5 authentication mechanisms: -- Option: esmtp-service SERVICE-NAME Sets the name of GSSAPI service. -- Option: esmtp-hostname HOSTNAME Sets hostname of the machine. -- Option: esmtp-generic-service SERVISE-NAME Sets generic service name. -- Option: esmtp-passcode PASSCODE Sets passcode. -- Option: esmtp-realm REALM-NAME Sets GSSAPI realm. Following option is useful with ANONYMOUS authentication mechanism: -- Option: esmtp-anonymous-token TOKEN Sets the token to be used with ANONYMOUS authentication mechanism 5.2.5 Encryption Settings ------------------------- -- Option: ssl YES-OR-NO This option enables the TLS/SSL encryption between the MUA and the MTA. Value `no' is the default, but using the TLS/SSL encryption is recommended. You should also specify a private key and a certificate using the `ssl-key' and `ssl-cert' keywords (defined below). *Note TLS/SSL::, for details. -- Option: ssl-oneway YES-OR-NO This option enables the "ONEWAY" encryption. Use this mode, when you want to use the TLS/SSL, but your MUA doesn't provide a support for ESMTP TLS/SSL. Using this option doesn't require using the `ssl-key' and `ssl-cert' keywords. -- Option: ssl-cert FILE-NAME Specify a certificate for the TLS/SSL encryption. Value `anubis.pem' is the default. -- Option: ssl-key FILE-NAME Specify a private key for the TLS/SSL encryption. Value `anubis.pem' is the default. -- Option: ssl-cafile FILE-NAME Specify a CA certificate file (supported only by GnuTLS). 5.2.6 Security Settings ----------------------- The following options control various security settings. -- Option: allow-local-mta YES-OR-NO For security reasons, this option is set to `no', but the `yes' value enables the `local-mta' keyword (or `--local-mta' command line option), so if you want to use a local mail server, which works on standard input and output, a supervisor must set this option to `yes'. The option is available only in system configuration file. -- Option: drop-unknown-user YES-OR-NO This option drops an unknown user, i.e. a client which has not been verified by IDENT service. Value `no' is the default. -- Option: user-notprivileged USERNAME For security reasons, it is recommended to create an unprivileged user, which the server runs as most of the time, when doing unprivileged operations. The option is available only in system configuration file. For example: user-notprivileged "anubis.unprivileged" *Caution:* Create a user account named `anubis.unprivileged' in the `/etc/passwd', if necessary. Add this user name also to the `/etc/anubis.allow', if using GNU Anubis with PAM support. -- Option: rule-priority VALUE This statement defines the order of execution of the system and user `RULE' sections (*Note Rule System::, for detailed description). It is available only in system configuration file. `system' The system section is executed first, then the user section is executed. `user' The user section is executed first, next the system section is executed. `system-only' Only the system `RULE' section is executed. `user-only' Only the user `RULE' section is executed. -- Option: control-priority VALUE Sets the order of processing the `CONTROL' sections. The option is available only in system configuration file. Its possible values are: `system' The system `CONTROL' section is processed first. Notice, that this means that the user may override the system settings in his configuration file. This is the default setting. `user' The user `CONTROL' section is processed first. Thus, the system-wide settings always override the user private settings. 5.3 TRANSLATION Section ======================= The `TRANSLATION' section specifies how to translate remote or local user names, or host names or addresses, to local user names. The `TRANSLATION' section is available _only_ in the system configuration file. Syntax: ---BEGIN TRANSLATION--- translate [USER@]ADDRESS into USERNAME ... ---END--- ADDRESS means host name or IP address. You can also specify `0.0.0.0', and it means any address (`INADDR_ANY'). An example: ---BEGIN TRANSLATION--- translate jack@somewhere.net into john ---END--- The rule above will allow a remote user `jack' at `somewhere.net' to use the configuration file of the local user `john'. Or you can write: `translate somewhere.net into john', and this means that _all_ users at `somewhere.net' are allowed to use the local john's configuration file. 5.4 GUILE Section ================= -- Command: guile-output FILE Specifies the name of the file to bind to the Scheme standard error and output ports. This option has no effect if GNU Anubis is started with either of `--foreground' or `--stdio' command line options. -- Command: guile-debug YES-OR-NO When set to `yes' enables Guile stack traces and debugging output. -- Command: guile-load-path-append PATH Appends the given PATH to the list of Guile load paths (*note %load-path: (guile)Build Config.). -- Command: guile-load-program FILE Reads the given Scheme program. 6 The Rule System ***************** The rule system is a core part of GNU Anubis. It can be regarded as a program that is executed for every outgoing message. Throughout this chapter, when showing syntax definitions, the optional parts of these will be enclosed in a pair of square brackets, e.g.: keyword [OPTIONAL-PART] MANDATORY-PART When the square braces are required symbols, they will be marked as such, e.g.: remove `['KEY`]' The rule system is defined in "RULE" section. The statements within this section are executed sequentially. Each statement is either an "action" or a "conditional statement". 6.1 Actions =========== An "action" is a statement defining an operation to be performed over the message. Syntactically, each action is COMMAND [=] RIGHT-HAND-SIDE Where COMMAND specifies a particular operation and RIGHT-HAND-SIDE specifies the arguments for it. The equal sign is optional. 6.2 Conditional Statements ========================== A "conditional statement" defines the control flow in a section. It allows to execute arbitrary actions depending on whether a certain condition is met. The conditional statement in its simplest form is: if CONDITION ACTION-LIST-1 fi If CONDITION evaluates to true, then the list of statements ACTION-LIST-1 is executed. A simple CONDITION has the following syntax: PART [SEP] [OP] [PATTERN-MATCH-FLAGS] REGEX (where the square brackets denote optional parts). Its parts are: PART The PART specifies which part of the input should be considered when evaluating the condition. It is either `command', meaning the text of an smtp command issued while sending the message, or `header', meaning the value of an RFC822 header. Either of the two may be followed by the name of the corresponding command or header enclosed in square brackets. If this part is missing, all command or headers will be searched. SEP Optional "concatenation separator". *Note Concatenations::, for its meaning. OP Either `=', meaning "match", or `!=', meaning "does not match". Missing OP is equivalent to `='. PATTERN-MATCH-FLAGS Optional PATTERN-MATCH-FLAGS alter the pattern matching type used in subsequent conditional expression. It will be described in detail in the section *note Regular Expressions::. REGEX The REGEX is a regular expression enclosed in double quotes. The condition yields true if the requested part of the input matches the given regular expression (if OP is `='), or does not match it (if OP is `!='). The simplest example: if header [Subject] "^ *Re:" ... fi The actions represented by ... will be executed only if the `Subject:' header of the message starts with `Re:' optionally preceded by any amount of whitespace. The more elaborate form of a conditional allows you to choose among the two different action sets depending on a given condition. The syntax is: if CONDITION ACTION-LIST-1 else ACTION-LIST-2 fi Here, the ACTION-LIST-1 is executed if the CONDITION is met. Otherwise, ACTION-LIST-2 is executed. if CONDITION ACTION-LIST-1 else ACTION-LIST-2 fi Note also, that in the examples above any of the statements ACTION-LIST may contain conditionals, so that the conditional statements may be nested. This allows to create very sophisticated rule sets. As an example, consider the following statement: if [List-Id] :re ".*" modify [Subject] "[Anubis Commit Notice] &" else if [List-Id] :re ".*" modify [Subject] "[Anubis Bug Notice] &" else add [X-Passed] "Subject checking" fi fi This statement, depending on the value of `List-Id' header, will prepend the `Subject' header with an identification string, or add an `X-Passed' header if no known `List-Id' was found. 6.2.1 Concatenations -------------------- It is important to understand that conditional expressions choose the first match. To illustrate this, lets suppose you need to store values of all recipients from the envelope in the `X-Also-Delivered-To' header. A naive way to do so is: if command [rcpt to:] = "(.*)" add header [X-Also-Delivered-To] "\1" fi However, this will store only the very first `RCPT TO' value, so you will not achieve your goal. To help you in this case, `anubis' offers a "concatenation" operator, whose effect is to concatenate the values of all requested keys prior to matching them against the regular expression. Syntactically, the concatenation operator is a string enclosed in parentheses, placed right after the key part of a condition. This string is used as a separator when concatenating values. For example: if command [rcpt to:] (",") = "(.*)" add header [X-Also-Delivered-To] "\1" fi This fragment will first create a string consisting of all `RCPT TO' addresses, separated by a comma, and then will match it against the regular expression on the right hand side. Since this expression matches any string, the `\1' will contain a comma-separated list of addresses. 6.3 Triggers ============ Triggers are conditional statements that use the value of the `Subject' header to alter the control flow. Syntactically, a trigger is: trigger [FLAGS] PATTERN ACTION-LIST done Here, PATTERN is the pattern against which the `Subject' header is checked, FLAGS are optional flags controlling the type of regular expression used (*note Regular Expressions::). For backward compatibility, the keyword `rule' may be used instead of `trigger'. The triggers act as follows: First, the value of the `Subject' header is matched against the pattern `@@'PATTERN. If it matches, then the matched part is removed from the `Subject', and the ACTION-LIST is executed. Basically, putting aside the possibility to use different flavors of regular expressions, a trigger is equivalent to the following statement: if header[Subject] :posix "(.*)@@PATTERN" modify header [Subject] "\1" ACTION-LIST fi Thus, adding the `@@RULE-NAME' code to the `Subject' header of your message, triggers a rule named RULE-NAME, specified in a user configuration file. For example: ---BEGIN RULE--- trigger :basic "^gpg-encrypt-john" gpg-encrypt "john's_gpg_key" done ---END--- Now you can simply send an email with the following subject: `hello John!@@gpg-encrypt-john' to process an outgoing message with the rule specified above--encrypt message with a John's public key. Moreover, the trigger will remove the `@@', so John will only receive a message with a subject `hello John!'. Another example shows an even more dynamic trigger, that is using a substitution and back-references: ---BEGIN RULE--- trigger :extended "^gpg-encrypt:(.*)" gpg-encrypt "\1" add [X-GPG-Comment] "Encrypted for \1" done ---END--- To encrypt a message to user e.g. `John', simply send an email with a subject `hello John!@@gpg-encrypt:john's_gpg_key'. This way, you decide at a run time which public key should be used, without creating separate rules for each user; thanks to back-references, those 3--4 lines are enough. 6.4 Boolean Operators ===================== The following table lists the three boolean operators that can be used in Anubis conditional expressions in the order of increasing binding strength: * `OR' * `AND' * `NOT' As an example, let's consider the following statement: if header[X-Mailer] "mutt" or header[X-Mailer] "mail" \ and not header[Content-Type] "^multipart/mixed;.*" ACTION fi In this case the ACTION will be executed if the `X-Mailer' header contains the word `mutt'. The same ACTION will also be executed if the `X-Mailer' header contains the word `mail' _and_ the value of the `Content-Type' header does not begin with the string `multipart/mixed'. Now, if we wished to execute the ACTION for any message sent using `mail' or `mutt' whose `Content-Type' header does not begin with the string `multipart/mixed', we would write the following: if (header[X-Mailer] "mutt" or header[X-Mailer] "mail") \ and not header[Content-Type] "^multipart/mixed;.*" ACTION fi Notice the use of parentheses to change the binding strength of the boolean operators. 6.5 Regular Expressions ======================= GNU Anubis supports two types of regular expressions: POSIX (both basic and extended), and Perl-style regular expressions. Among this, the former are always supported, whereas the support for the latter depends on the configuration settings at compile time. The default type of regular expressions is POSIX Extended. Notice, that regular expressions often contain characters, prefixed with backslash (e.g. `\(' in basic POSIX or `\s' in perl-style regexp). Due to escape substitution (*note Configuration::), you will have to escape the backslash character, e.g. write: modify :perl body ["\\stext"] "text" instead of # WRONG! modify :perl body ["\stext"] "text" However, this rule does not apply to back references, i.e. `"\1"' is OK. A number of modifiers is provided to change the type of regular expressions. These are described in the following table. `:regex' `:re' Indicates that the following pattern should be considered a regular expression. The default type for this expression is assumed. `:perl' `:perlre' The regular expression is a Perl-style one. `:exact' `:ex' Disables regular expression matching, all patterns will be matched as exact strings. `:scase' Enables case-sensitive comparison. `:icase' Enables case-insensitive comparison. `:basic' Switches to the POSIX Basic regular expression matching. `:extended' Switches to the POSIX Extended regular expression matching. The special statement `regex' allows you to alter the default regular expression type. For example, the following statement regex :perl :scase sets the default regular expression types to Perl-style, case-sensitive. The settings of `regex' statement regard only those patterns that appear after it in the configuration file and have force until the next occurrence of the `regex' statement. A couple of examples: if header[Subject] :perlre "(?<=(?. Before reporting a bug, make sure you have actually found a real bug. Carefully reread the documentation and see if it really says you can do what you are trying to do. If it is not clear whether you should be able to do something or not, report that too; it's a bug in the documentation! Appendix A Pixie & Dixie ************************ * Introduction This document describes a new scheme for client authentication and authorization in GNU Anubis 4.x. * Task Description So far the only authentication method used by Anubis was based on the AUTH protocol (RFC 1413) (ftp://ftp.rfc-editor.org/in-notes/rfc1413.txt), and thus required client party to use a popular daemon `identd', which listens on TCP port 113 for authentication requests. As its primary advantage, this method allows to quickly identify whom the server had to deal with, i.e. to obtain user name or his UID. Actually, the authentication process finishes before the client sends over his first byte. Besides, this method allows to process the entire SMTP envelope. It has, however, several drawbacks, first of them being the requirement to run `identd' on the client machine, which is not always possible (e.g. on mobile devices), and may be considered harmful for the system security (due to sending user ID over the wire). * The Proposed Solution Proposed are two operation modes: 1. "Traditional" or "transparent" (also known as "Pixie" ;-) 2. "Authentication first" (also known as "Dixie" ;-) A short description of each mode follows: - `Pixie' mode - Server requires the remote party to authenticate itself using SMTP AUTH (RFC 2554) (ftp://ftp.rfc-editor.org/in-notes/rfc2554.txt). - Early processing of SMTP envelope is possible. - Connections between MUA and MTA are tunneled "on the fly" - `Dixie' mode In this mode GNU Anubis runs its own user database, additionally translating logins (*note login translation::). It also is able to keep users' configuration files (an additional option and an advantage -- *note anubis database::). Users are authenticated using ESMTP AUTH protocol. Early processing of SMTP envelope is not possible in this mode , instead it becomes possible only after the authentication is finished successfully. This mode also delays connecting to the MTA, since Anubis first has to perform ESMTP AUTH, and only after finishing authentication, does it read and process the user's configuration file and connects to the selected MTA. Of course, the client is not able to begin sending messages until he is authenticated and accepted by Anubis. * Details There is a great difference between the two modes. To begin with, `Pixie' mode provides a tunnel (or proxy), in the sense that Anubis connects user's MUA to the remote MTA without requiring any special actions from the user. Let's consider a simple interaction between `Machine-A', which runs Anubis 4, and `Machine-B', where MUA is run. A: 220 Machine-A (GNU Anubis vX.X [Dixie]) ESMTP time; send your identity! B: EHLO Machine-B A: 250-Machine-A Hello ID 250-STARTTLS 250-AUTH DIGEST-MD5 CRAM-MD5 LOGIN 250-XDATABASE 250 HELP B: STARTTLS A: 220 2.0.0 Ready to start TLS B: AUTH [method-specific authentication interchange follows] Now, the Anubis server has authenticated the client using data from Anubis database! I'd like this database to contain, beside the user name and password, the name and password of this user on Machine-A. Confusing? Let's suppose that the database contains following record: JohnSmith encrypted-pass-1 John The user has authenticated himself as `JohnSmith' with password `encrypted-pass-1', using ESMTP AUTH, and the given credentials matched those from the Anubis database. Now, Anubis, which has been running with super-user privileges, switches to UID of the user `John'. Such solution will allow for a very flexible database, that would ease the administration tasks, since users will be able to update their corresponding records (of course, if the system administrator grants them such privileges). For instance, ODBC, SQL? Let's return to our sample session. After successful authentication and switching to the user's privileges, Anubis parses file `~/.anubisrc'. Then, based on user's configuration settings, it connects to the MTA and from then on operates as SMTP tunnel and mail processor :-). It sends the following response to `Machine-B': A: 220 OK, Welcome. Continue sending your mail! * Further details The above description shows that it is impossible to use both `Pixie' and `Dixie' simultaneously. It is the responsibility of the system administrator to decide which operation mode to use. We could probably provide for a smooth switching between the two modes, without requiring to restart the daemon... However, it is not critical. Restarting the daemon in order to switch to another operation mode is also a feasible solution. Now, let me describe for what kind of users each mode is intended. The traditional (`Pixie') mode is intended for those users who use Anubis on a single machine or within a local network that allows to use `identd'. In short, `Pixie' is useful when the use of `identd' is possible and safe. In contrast, the new mode `Dixie' is intended for more complex setups, where a single machine running GNU Anubis serves a number of clients connecting from different machines and networks. It is supposed that no client machine is running `identd'. The only recommendation for this mode is that each user have a system account on the machine running Anubis. But then, even this is not required! That's a feature I haven't described yet :^) As described above, Anubis database must contain second login name in order for Anubis to be able to switch to the user's privileges and parse his `~/.anubisrc' file. Now, I supposed that the database is able to keep user configuration files as well. So, each database record must contain an additional flag informing Anubis whether it should read the local file `~/.anubisrc', or read the configuration file stored in the database. Sure enough, GNU Anubis still will have to switch to the user's privileges, for security reasons, but this can be done using usual `user-notprivileged' configuration (*note user-notprivileged: Security Settings.). Surely you have noticed that in its response to EHLO command Dixie returned `250-XDATABASE' capability. Yes, this is exactly that command that I'd like to be used for remote management of the database records (after having successfully passed ESMTP AUTH). Available operations are: `ADD', `MODIFY', `REMOVE', meaning addition, modification and removal of a user record, and `UPLOAD', providing a way to upload the user's configuration file `~/.anubisrc'. This solution will free the users from the obligation to have `~/.anubisrc' on the server machine, so they, for the first time since early Anubis versions, will be able to have their _own_ configuration files. Current versions () require that the user configuration file be stored on the server machine before the user is able to use the service. This approach requires a certain attention from the system administrator. Should the user wish to change something in his configuration file, he would have to install the modified file on `Machine-A' (that's how it works now, and that's how it will continue to work for `Pixie' mode). The new `Dixie' mode solves this and frees the user from necessity to contact the system administrator of `Machine-A'. The Anubis database engine is supposed to check the correctness of the uploaded configuration file and inform the client about the result. It also should compute MD5 hash of the file and compare it to the one sent by the user... What for? * A program sending user's configuration file Well, we're almost finished. The user will have a small program, `config-sender', written in whatever language (C, Java, C#), whose main purpose is to send user's configuration file to the database. Such a program could even be installed on a mobile device! Notice also, that this program is optional, the user is not required to use it. I envision a situation where: 1. A user logs in to his account on `Machine-B' 2. His `~/.profile' invokes `config-sender' program. This program, in turn, computes MD5 sum of the local `~/.anubisrc' file and sends it to Anubis. There it will be compared to the sum kept in the Anubis database, and if the two sums differ, the `config-sender' will upload the contents of `~/.anubisrc'... (1) 3. The `config-sender' program will, of course, connect to the Anubis database using ESMTP (TLS/AUTH) and `XDATABASE'. Such a program will be an additional advantage, since no existing MUA is, of course, able to use `XDATABASE' command to manage Anubis database. Notice however, that GNU Hydrant (http://savannah.gnu.org/projects/hydrant) will probably support `XDATABASE' in the future... * The End. Thus, the user will simply use his MUA, no identd, no hassle :) Actually, the only requirement for the MUA is that it support ESMTP AUTH. Unfortunately, some MUA, even on UNIX-like systems, are still not able to use ESMTP AUTH. But in this case, the user can install Anubis on his machine and use it to perform authentication ;-))) And the last detail: what to do if the remote MTA also requires ESMTP AUTH? The answer is quite simple: GNU Anubis is already able to handle this (*note esmtp-auth: Basic Settings.). * Summary (`Dixie' mode) - a little slower than `Pixie', in the sense that the actual connection to the MTA is established only after successful authentication - does not require `identd'! - allows the user full control over his configuration settings - delays processing of SMTP envelope until after successful authentication. * PS: A couple of words about storing configuration files in the database... These can be stored in a special directory as usual files, then each database record will have an additional field with the name of the configuration file for the given user. -- THE END -- ---------- Footnotes ---------- (1) The scheme implemented currently is a bit different. First, the `config-sender' program issues an `EXAMINE' command that fetches the contents of the user configuration file from the server. Then, it compares it with the local copy kept on the client machine. If the copies differ, `config-sender' issues `UPLOAD' and thus updates the configuration on the server. Appendix B Multi-Part Message Processing **************************************** 0. PREFACE In its current state (as of Anubis version 4.1.1) Anubis has proven to be a useful tool for processing plain text outgoing messages. However, its use with MIME messages creates several problems despite of a flexible ruleset supported by the program. This RFC proposes a new mode of operation that should make processing of MIME messages more convenient. 1. INTRODUCTION In general, Anubis processes a message using a set of user-defined rules, called "user program", consisting of "conditional statements" and "actions". Both of them may operate on message body as well as on its headers. This mode of operation suites excellently for plain text messages, however it does have its drawbacks when processing multi-part messages. To begin with, only the first part of multi-part messages is processed, the rest of message is usually passed to the MTA verbatim. Thus, this part can be processed by the user program only if it is in plain text: parts encoded by quoted-printable or, worse yet, base-64 encoding cannot be processed this way. The only way for the user to process non plain-text multi-part messages is by using some extension procedures (usually external scripts). A special configuration setting `read-entire-body' (*note Basic Settings::) is provided that forces Anubis to process the entire body of a multi-part message (among other effects it means passing entire body to the external scripts as well). However, it does not help solve the problem, since no attempt is being made to decode parts of the message, so the user is left on his own when processing such messages. The solution proposed by this memo boils down to the following: process each part of the multi-part message as a message on its own allowing user to define different RULE sections for processing different MIME types. The following sections describe the approach in more detail. 2. MULTI-PART MESSAGE PROCESSING When processing a multi part message, Anubis first determines its MIME type. A user is allowed to define several RULE sections(1) that are supposed to handle different MIME types. Anubis keeps a `type <-> section' association table (a "dispatcher table") which is used to determine the entry point for processing of each particular part. If the dispatcher table does not contain an entry for the given MIME type, the contents of the part is passed verbatim. Otherwise, Anubis decodes the part body and passes it for further processing to the RULE section. When invoking this particular section, MIME headers act as a message headers and MIME body acts as its body. After the code section finishes processing of the message part, it is encoded again(2) and then passed to the output. 3. RECURSIVE NATURE MIME standards allow multi-part messages to be nested to arbitrary depth, therefore the described above process is inherently recursive. This brings following implications: 1. The dispatcher table must contain several built-in entries that will handle recursive descent to the messages of determined MIME type. At least messages having `multipart/*' and `message/rfc822' contents must be handled. These entries must be configurable, thus giving final user a possibility to disable some of any of them. Preferably there should exist a way of specifying new recursive types as well. 2. A confuguration parameter must be provided that will limit the maximum recursion depth for such messages. 4. MIME DISPATCHER TABLE The structure of MIME dispatcher table should allow for flexible search of user program entries depending on MIME type of the part being processed. It is important also that it allows for a "default entry", i.e. an entry that will be used for processing a part whose type is not explicitely mentioned in the table. An absence of such default entry should be taken as indication that the part must be transferred verbatim. Thus, each entry of the dispatcher table must contain at least following members. `type' Specifies regular expressions describing MIME type this entry handles. For the sake of clarity this memo uses shell-style regular expressions (see `glob(7)' or `fnmatch(3)'). However, Anubis implementation may use any other regular expression style it deems appropriate. `entry point' Specifies an entry point to the code section that handles MIME parts of given type. The entry point is either `nil', meaning default processing (thus the default entry can be represented as `("*" . nil)' _at the end of the table_), or one of predefined entry points serving for recursive procession of message parts, or, finally, it is a code index of a user-defined rule section. Dispatcher table may contain several entries matching a given MIME type. In this case, the `entry point' of each of them must be invoked in turn. For example, consider this dispatcher table: => `plaintext' => `patchfile' => `anytext' When processing a part of type `text/plain' using this dispatcher table, first section named `plaintext' is called, then its output is gathered and used as input while calling section named `anytext'. Such approach allows for building flexible structured user programs. 5. CONFIGURATION ENTITIES This memo proposes addition of following configuration entities to `CONTROL' section of Anubis configuration file. These entries may be used in both system-wide and user-specific configuration files, the order of their priority being determined as usual by `rule-priority' statement (*note Security Settings::). -- Option: clear-dispatch-table This option discards from the dispatcher table all entries gathered so far. -- Option: dispatch-mime-type SECTION-ID REGEXP-LIST This option adds or modifies entries in MIME dispatcher table. SECTION-ID specifies the "section identifier", i.e. either the name of a user-defined rule section, or one of the keywords `none' and `recurse'. In the former case, Anubis must make sure the named section is actually defined in the configuration file and issue an error message otherwise. REGEXP-LIST is whitespace-separated list of regular expressions specifying MIME types that are to be handled by SECTION-ID. The effect of this option is that for each regular expression RE from the list REGEXP-LIST, the dispatcher table is searched for an entry whose `type' field is exactly the same as RE(3). If such an entry is found, its `entry code' field is replaced with SECTION-ID. Otherwise, if no matching entry was found a new one is constructed: (SECTION-ID . RE) and appended to the end of the list. For example: dispatch-mime-type recurse "multipart/*" "message/rfc822" dispatch-mime-type Text "text/*" dispatch-mime-type none "*" This example specifies that messages (or parts) with types matching `multipart/*' and `message/rfc822' must be recursed into, those of type `text/*' must be processed by user-defined section `Text' and the rest of parts must be transferred verbatim. The section `Text' must be declared somewhere in the configuration file as BEGIN Text ... END Notice that the very first `dispatch-mime-type' specifies a built-in entry. This memo does not specify whether such a built-in entry must be present by default, or it should be explicitely declared as in the example above. The explicit declaration seems to have advantage of preserving backward compatibility with versions 4.0 and earlier of Anubis (*note COMPATIBILITY CONSIDERATIONS::). Notice also that when encountering the very first `dispatch-mime-type' (or `dispatch-mime-type-prepend', see below) statement _in the user configuration file_, Anubis must remove the default entry (if any) from the existing dispatcher table. Such entry should be added back after processing user's `CONTROL' section, unless `clear-dispatch-table' has been used. -- Option: dispatch-mime-type-prepend SECTION-ID REGEXP-LIST Has the same effect as `dispatch-mime-type' except that the entries are prepended to the dispatcher table. -- Option: recursion-depth NUMBER This option limits the maximum recursion depth when processing multi-part messages to NUMBER. 6. TEXT vs BINARY MIME PARTS This memo does not determine how exactly is Anubis supposed to discern between text and binary messages. The siplest way is possibly using `Content-Type' header: if it contains `charset=' then it describes a text part. Otherwise it describes a binary part. Probably some more sophisticated methods should be provided. To avoid dependency on any particular charset, text parts must be decoded to UTF-8. Correspondingly, any literals used in Anubis configuration files must represent valid UTF-8 strings. However, this memo does not specify whether Anubis implementation should enforce UTF-8 strings in its configuration files. It is possible to specify processing rules for binary MIME parts. However, Anubis does not provide any mechanism for binary processing, not is it supposed to provide any. This memo maintains that the existing `external-body-processor' and `guile-process' statements are quite sufficient for processing any binary message parts. 7. SAMPLE CONFIGURATION FILE BEGIN CONTROL dispatch-mime-type recurse "multipart/*" "message/rfc822" dispatch-mime-type plaintext "text/plain" dispatch-mime-type image "img/*" END CONTROL SECTION plaintext modify body ["now"] "then" END SECTION image external-body-processor resize-message END This sample configuration shows the idea of using `external-body-processor' statement for binary part processing. Following version of `resize-message' script uses `convert' program for reducing image size to 120x120 pixels: #! /bin/sh TMP=$HOME/tmp/$$ cat - > $TMP convert -size 120x120 $TMP.jpg -resize 120x120 +profile '*' out-$TMP rm $TMP cat out-$TMP rm out-$TMP 8. COMPATIBILITY CONSIDERATIONS In the absense of any `dispatch-mime-type' statements, Anubis should behave exactly as version 4.0 did. Specifying clear-dispatch-table in the user configuration file should produce the same effect. This can be useful if system-wide configuration file contained some `dispatch-mime-type' statements. 9. SECURITY CONSIDERATIONS This specification is believed to not introduce any special security considerations. ---------- Footnotes ---------- (1) This is already possible, *Note Call Action::. (2) Note that the code section could have modified the `Content-Type' header and, particularly, its `encoding' part, therefore it is not necessary that the resulting part is encoded using the same method as the original one (3) Byte-for-byte comparison Appendix C GNU Free Documentation License ***************************************** Version 1.2, November 2002 Copyright (C) 2000,2001,2002 Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. 0. PREAMBLE The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others. This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software. We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference. 1. APPLICABILITY AND DEFINITIONS This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law. A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language. A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them. The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none. The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words. A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque". Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only. The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text. A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition. The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License. 2. VERBATIM COPYING You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3. You may also lend copies, under the same conditions stated above, and you may publicly display copies. 3. COPYING IN QUANTITY If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects. If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages. If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public. It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document. 4. MODIFICATIONS You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version: A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission. B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement. C. State on the Title page the name of the publisher of the Modified Version, as the publisher. D. Preserve all the copyright notices of the Document. E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices. F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below. G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice. H. Include an unaltered copy of this License. I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence. J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission. K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein. L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles. M. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version. N. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section. O. Preserve any Warranty Disclaimers. If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles. You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard. You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one. The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version. 5. COMBINING DOCUMENTS You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers. The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a