26. The appendfile transport

The appendfile transport delivers a message by appending it to an existing file, or by creating an entirely new file in a specified directory. Single files to which messages are appended can be in the traditional Unix mailbox format, or optionally in the MBX format supported by the Pine MUA and University of Washington IMAP daemon, inter alia. When each message is being delivered as a separate file, "maildir" format can optionally be used to give added protection against failures that happen part-way through the delivery. A third form of separate-file delivery known as "mailstore" is also supported. For all file formats, Exim attempts to create as many levels of directory as necessary, provided that create_directory is set.

The code for the optional formats is not included in the Exim binary by default. It is necessary to set SUPPORT_MBX, SUPPORT_MAILDIR and/or SUPPORT_MAILSTORE in `Local/Makefile' to have the appropriate code included.

Exim recognises system quota errors, and generates an appropriate message. Exim also supports its own quota control within the transport, for use when the system facility is unavailable or cannot be used for some reason.

If there is an error while appending to a file (for example, quota exceeded or partition filled), Exim attempts to reset the file's length and last modification time back to what they were before. If there is an error while creating an entirely new file, the new file is removed.

Before appending to a file, a number of security checks are made, and the file is locked. A detailed description is given below, after the list of private options.

The appendfile transport is most commonly used for local deliveries to users' mailboxes. However, it can also be used as a pseudo-remote transport for putting messages into files for remote delivery by some means other than Exim. "Batch SMTP" format is often used in this case (see the use_bsmtp option).

26.1 The file and directory options

The file option specifies a single file, to which the message is appended; the directory option specifies a directory, in which a new file containing the message is created. Only one of these two options can be set, and for normal deliveries to mailboxes, one of them must be set.

However, appendfile is also used for delivering messages to files or directories whose names (or parts of names) are obtained from alias, forwarding, or filtering operations (for example, a save command in a user's Exim filter). When such a transport is running, $local_part contains the local part that was aliased or forwarded, and $address_file contains the name (or partial name) of the file or directory generated by the redirection operation. There are two cases:

• If neither file nor directory is set, the redirection operation must specify an absolute path (one that begins with `/'). This is the most common case when users with local accounts use filtering to sort mail into different folders. See for example, the address_file transport in the default configuration. If the path ends with a slash, it is assumed to be the name of a directory. A delivery to a directory can also be forced by setting maildir_format or mailstore_format.
• If file or directory is set for a delivery from a redirection, it is used to determine the file or directory name for the delivery. Normally, the contents of $address_file are used in some way in the string expansion.

As an example of the second case, consider an environment where users do not have home directories. They may be permitted to use Exim filter commands of the form:

save folder23

or Sieve filter commands of the form:

require "fileinto";
fileinto "folder23";

In this situation, the expansion of file or directory in the transport must transform the relative path into an appropriate absolute file name. In the case of Sieve filters, the name inbox must be handled. It is the name that is used as a result of a "keep" action in the filter. This example shows one way of handling this requirement:

file = ${if eq{$address_file}{inbox} \
            {/var/mail/$local_part} \
            {${if eq{${substr_0_1:$address_file}}{/} \
                  {$address_file} \
                  {$home/mail/$address_file} \
            }} \
       }

With this setting of file, inbox refers to the standard mailbox location, absolute paths are used without change, and other folders are in the `mail' directory within the home directory.

Note 1: While processing an Exim filter, a relative path such as `folder23' is turned into an absolute path if a home directory is known to the router. In particular, this is the case if check_local_user is set. If you want to prevent this happening at routing time, you can set router_home_directory empty. This forces the router to pass the relative path to the transport.

Note 2: An absolute path in $address_file is not treated specially; the file or directory option is still used if it is set.

26.2 Private options for appendfile

Setting this option permits delivery to named pipes (FIFOs) as well as to regular files. If no process is reading the named pipe at delivery time, the delivery is deferred.

By default, appendfile will not deliver if the path name for the file is that of a symbolic link. Setting this option relaxes that constraint, but there are security issues involved in the use of symbolic links. Be sure you know what you are doing if you set this. Details of exactly what this option affects are included in the discussion which follows this list of options.

See the description of local delivery batching in chapter Address batching in local transports. However, batching is automatically disabled for appendfile deliveries that happen as a result of forwarding or aliasing or other redirection directly to a file.

See the description of local delivery batching in chapter Address batching in local transports.

When this option is set, the group owner of the file defined by the file option is checked to see that it is the same as the group under which the delivery process is running. The default setting is false because the default file mode is 0600, which means that the group is irrelevant.

When this option is set, the owner of the file defined by the file option is checked to ensure that it is the same as the user under which the delivery process is running.

As appendfile writes the message, the start of each line is tested for matching check_string, and if it does, the initial matching characters are replaced by the contents of escape_string. The value of check_string is a literal string, not a regular expression, and the case of any letters it contains is significant.

If use_bsmtp is set the values of check_string and escape_string are forced to "." and ".." respectively, and any settings in the configuration are ignored. Otherwise, they default to "From " and ">From " when the file option is set, and unset when any of the directory, maildir, or mailstore options are set.

The default settings, along with message_prefix and message_suffix, are suitable for traditional "BSD" mailboxes, where a line beginning with "From " indicates the start of a new message. All four options need changing if another format is used. For example, to deliver to mailboxes in MMDF format:

check_string = "\1\1\1\1\n"
escape_string = "\1\1\1\1 \n"
message_prefix = "\1\1\1\1\n"
message_suffix = "\1\1\1\1\n"

When this option is true, Exim attempts to create any missing superior directories for the file that it is about to write. A created directory's mode is given by the directory_mode option.

The group ownership of a newly created directory is highly dependent on the operating system (and possibly the file system) that is being used. For example, in Solaris, if the parent directory has the setgid bit set, its group is propagated to the child; if not, the currently set group is used. However, in FreeBSD, the parent's group is always used.

This option constrains the location of files and directories that are created by this transport. It applies to files defined by the file option and directories defined by the directory option. In the case of maildir delivery, it applies to the top level directory, not the maildir directories beneath.

The option must be set to one of the words "anywhere", "inhome", or "belowhome". In the second and third cases, a home directory must have been set for the transport. This option is not useful when an explicit file name is given for normal mailbox deliveries. It is intended for the case when file names are generated from users' `.forward' files. These are usually handled by an appendfile transport called address_file. See also file_must_exist.

This option is mutually exclusive with the file option, but one of file or directory must be set, unless the delivery is the direct result of a redirection (see section The file and directory options).

When directory is set, the string is expanded, and the message is delivered into a new file or files in or below the given directory, instead of being appended to a single mailbox file. A number of different formats are provided (see maildir_format and mailstore_format), and see section Operational details for delivery to a new file for further details of this form of delivery.

When directory is set, but neither maildir_format nor mailstore_format is set, appendfile delivers each message into a file whose name is obtained by expanding this string. The default value generates a unique name from the current time, in base 62 form, and the inode of the file. The variable $inode is available only when expanding this option.

If appendfile creates any directories as a result of the create_directory option, their mode is specified by this option.

See check_string above.

This option is mutually exclusive with the directory option, but one of file or directory must be set, unless the delivery is the direct result of a redirection (see section The file and directory options). The file option specifies a single file, to which the message is appended. One or more of use_fcntl_lock, use_flock_lock, or use_lockfile must be set with file.

If you are using more than one host to deliver over NFS into the same mailboxes, you should always use lock files.

The string value is expanded for each delivery, and must yield an absolute path. The most common settings of this option are variations on one of these examples:

file = /var/spool/mail/$local_part
file = /home/$local_part/inbox
file = $home/inbox

In the first example, all deliveries are done into the same directory. If Exim is configured to use lock files (see use_lockfile below) it must be able to create a file in the directory, so the "sticky" bit must be turned on for deliveries to be possible, or alternatively the group option can be used to run the delivery under a group id which has write access to the directory.

This option requests the transport to check the format of an existing file before adding to it. The check consists of matching a specific string at the start of the file. The value of the option consists of an even number of colon-separated strings. The first of each pair is the test string, and the second is the name of a transport. If the transport associated with a matched string is not the current transport, control is passed over to the other transport. For example, suppose the standard local_delivery transport has this added to it:

file_format = "From       : local_delivery :\
               \1\1\1\1\n : local_mmdf_delivery"

Mailboxes that begin with "From" are still handled by this transport, but if a mailbox begins with four binary ones followed by a newline, control is passed to a transport called local_mmdf_delivery, which presumably is configured to do the delivery in MMDF format. If a mailbox does not exist or is empty, it is assumed to match the current transport. If the start of a mailbox doesn't match any string, or if the transport named for a given string is not defined, delivery is deferred.

If this option is true, the file specified by the file option must exist. A temporary error occurs if it does not, causing delivery to be deferred. If this option is false, the file is created if it does not exist.

By default, the appendfile transport uses non-blocking calls to fcntl() when locking an open mailbox file. If the call fails, the delivery process sleeps for lock_interval and tries again, up to lock_retries times. Non-blocking calls are used so that the file is not kept open during the wait for the lock; the reason for this is to make it as safe as possible for deliveries over NFS in the case when processes might be accessing an NFS mailbox without using a lock file. This should not be done, but misunderstandings and hence misconfigurations are not unknown.

On a busy system, however, the performance of a non-blocking lock approach is not as good as using a blocking lock with a timeout. In this case, the waiting is done inside the system call, and Exim's delivery process acquires the lock and can proceed as soon as the previous lock holder releases it.

If lock_fcntl_timeout is set to a non-zero time, blocking locks, with that timeout, are used. There may still be some retrying: the maximum number of retries is

(lock_retries * lock_interval) / lock_fcntl_timeout

rounded up to the next whole number. In other words, the total time during which appendfile is trying to get a lock is roughly the same, unless lock_fcntl_timeout is set very large.

You should consider setting this option if you are getting a lot of delayed local deliveries because of errors of the form

failed to lock mailbox /some/file (fcntl)

This timeout applies to file locking when using flock() (see use_flock); the timeout operates in a similar manner to lock_fcntl_timeout.

This specifies the time to wait between attempts to lock the file. See below for details of locking.

This specifies the maximum number of attempts to lock the file. A value of zero is treated as 1. See below for details of locking.

This specifies the mode of the created lock file, when a lock file is being used (see use_lockfile and use_mbx_lock).

When a lock file is being used (see use_lockfile), if a lock file already exists and is older than this value, it is assumed to have been left behind by accident, and Exim attempts to remove it.

If this option is set, it is expanded, and the result is taken as the current number of files in the mailbox. It must be a decimal number, optionally followed by K or M. This provides a way of obtaining this information from an external source that maintains the data.

If this option is set, it is expanded, and the result is taken as the current size the mailbox. It must be a decimal number, optionally followed by K or M. This provides a way of obtaining this information from an external source that maintains the data. This is likely to be helpful for maildir deliveries where it is computationally expensive to compute the size of a mailbox.

If this option is set with the directory option, the delivery is into a new file, in the "maildir" format that is used by other mail software. When the transport is activated directly from a redirect router (for example, the address_file transport in the default configuration), setting maildir_format causes the path received from the router to be treated as a directory, whether or not it ends with `/'. This option is available only if SUPPORT_MAILDIR is present in `Local/Makefile'. See section Maildir delivery below for further details.

This option is relevant only when maildir_use_size_file is set. It defines a regular expression for specifying directories, relative to the quota directory (see quota_directory), that should be included in the quota calculation. The default value is:

maildir_quota_directory_regex = ^(?:cur|new|\..*)$

This includes the `cur' and `new' directories, and any maildir++ folders (directories whose names begin with a dot). If you want to exclude the `Trash' folder from the count (as some sites do), you need to change this setting to

maildir_quota_directory_regex = ^(?:cur|new|\.(?!Trash).*)$

This uses a negative lookahead in the regular expression to exclude the directory whose name is `.Trash'. When a directory is excluded from quota calculations, quota processing is bypassed for any messages that are delivered directly into that directory.

This option specifies the number of times to retry when writing a file in "maildir" format. See section Maildir delivery below.

This option applies only to deliveries in maildir format, and is described in section Maildir delivery below.

Setting this option true enables support for `maildirsize' files. Exim creates a `maildirsize' file in a maildir if one does not exist, taking the quota from the quota option of the transport. If quota is unset, the value is zero. See maildir_quota_directory_regex above and section Maildir delivery below for further details.

The value of this option is a regular expression. If it is unset, it has no effect. Otherwise, before a maildir delivery takes place place, the pattern is matched against the name of the maildir directory, that is, the directory containing the `new' and `tmp' subdirectories that will be used for the delivery. If there is a match, Exim checks for the existence of a file called `maildirfolder' in the directory, and creates it if it does not exist. See section Maildir delivery for more details.

If this option is set with the directory option, the delivery is into two new files in "mailstore" format. The option is available only if SUPPORT_MAILSTORE is present in `Local/Makefile'. See section Operational details for delivery to a new file below for further details.

This option applies only to deliveries in mailstore format, and is described in section Operational details for delivery to a new file below.

This option applies only to deliveries in mailstore format, and is described in section Operational details for delivery to a new file below.

This option is available only if Exim has been compiled with SUPPORT_MBX set in `Local/Makefile'. If mbx_format is set with the file option, the message is appended to the mailbox file in MBX format instead of traditional Unix format. This format is supported by Pine4 and its associated IMAP and POP daemons, by means of the c-client library that they all use.

Note: The message_prefix and message_suffix options are not automatically changed by the use of mbx_format. They should normally be set empty when using MBX format, so this option almost always appears in this combination:

mbx_format = true
message_prefix =
message_suffix =

If none of the locking options are mentioned in the configuration, use_mbx_lock is assumed and the other locking options default to false. It is possible to specify the other kinds of locking with mbx_format, but use_fcntl_lock and use_mbx_lock are mutually exclusive. MBX locking interworks with c-client, providing for shared access to the mailbox. It should not be used if any program that does not use this form of locking is going to access the mailbox, nor should it be used if the mailbox file is NFS mounted, because it works only when the mailbox is accessed from a single host.

If you set use_fcntl_lock with an MBX-format mailbox, you cannot use the standard version of c-client, because as long as it has a mailbox open (this means for the whole of a Pine or IMAP session), Exim will not be able to append messages to it.

The string specified here is expanded and output at the start of every message. The default is unset unless file is specified and use_bsmtp is not set, in which case it is:

message_prefix = "From ${if def:return_path{$return_path}\
  {MAILER-DAEMON}} $tod_bsdinbox\n"

The string specified here is expanded and output at the end of every message. The default is unset unless file is specified and use_bsmtp is not set, in which case it is a single newline character. The suffix can be suppressed by setting

message_suffix =

If the output file is created, it is given this mode. If it already exists and has wider permissions, they are reduced to this mode. If it has narrower permissions, an error occurs unless mode_fail_narrower is false. However, if the delivery is the result of a save command in a filter file specifing a particular mode, the mode of the output file is always forced to take that value, and this option is ignored.

This option applies in the case when an existing mailbox file has a narrower mode than that specified by the mode option. If mode_fail_narrower is true, the delivery is deferred ("mailbox has the wrong mode"); otherwise Exim continues with the delivery attempt, using the existing mode of the file.

If this option is true, the comsat daemon is notified after every successful delivery to a user mailbox. This is the daemon that notifies logged on users about incoming mail.

This option imposes a limit on the size of the file to which Exim is appending, or to the total space used in the directory tree when the directory option is set. In the latter case, computation of the space used is expensive, because all the files in the directory (and any sub-directories) have to be individually inspected and their sizes summed. (See quota_size_regex and maildir_use_size_file for ways to avoid this in environments where users have no shell access to their mailboxes).

As there is no interlock against two simultaneous deliveries into a multi-file mailbox, it is possible for the quota to be overrun in this case. For single-file mailboxes, of course, an interlock is a necessity.

A file's size is taken as its used value. Because of blocking effects, this may be a lot less than the actual amount of disk space allocated to the file. If the sizes of a number of files are being added up, the rounding effect can become quite noticeable, especially on systems that have large block sizes. Nevertheless, it seems best to stick to the used figure, because this is the obvious value which users understand most easily.

The value of the option is expanded, and must then be a numerical value (decimal point allowed), optionally followed by one of the letters K, M, or G, for kilobytes, megabytes, or gigabytes. If Exim is running on a system with large file support (Linux and FreeBSD have this), mailboxes larger than 2G can be handled.

Note: A value of zero is interpreted as "no quota".

The expansion happens while Exim is running as root, before it changes uid for the delivery. This means that files that are inaccessible to the end user can be used to hold quota values that are looked up in the expansion. When delivery fails because this quota is exceeded, the handling of the error is as for system quota failures.

By default, Exim's quota checking mimics system quotas, and restricts the mailbox to the specified maximum size, though the value is not accurate to the last byte, owing to separator lines and additional headers that may get added during message delivery. When a mailbox is nearly full, large messages may get refused even though small ones are accepted, because the size of the current message is added to the quota when the check is made. This behaviour can be changed by setting quota_is_inclusive false. When this is done, the check for exceeding the quota does not include the current message. Thus, deliveries continue until the quota has been exceeded; thereafter, no further messages are delivered. See also quota_warn_threshold.

This option defines the directory to check for quota purposes when delivering into individual files. The default is the delivery directory, or, if a file called `maildirfolder' exists in a maildir directory, the parent of the delivery directory.

This option applies when the directory option is set. It limits the total number of files in the directory (compare the inode limit in system quotas). It can only be used if quota is also set. The value is expanded; an expansion failure causes delivery to be deferred. A value of zero is interpreted as "no quota".

See quota above.

This option applies when one of the delivery modes that writes a separate file for each message is being used. When Exim wants to find the size of one of these files in order to test the quota, it first checks quota_size_regex. If this is set to a regular expression that matches the file name, and it captures one string, that string is interpreted as a representation of the file's size. The value of quota_size_regex is not expanded.

This feature is useful only when users have no shell access to their mailboxes - otherwise they could defeat the quota simply by renaming the files. This facility can be used with maildir deliveries, by setting maildir_tag to add the file length to the file name. For example:

maildir_tag = ,S=$message_size
quota_size_regex = ,S=(\d+)

An alternative to $message_size is $message_linecount, which contains the number of lines in the message.

The regular expression should not assume that the length is at the end of the file name (even though maildir_tag puts it there) because maildir MUAs sometimes add other information onto the ends of message file names.

See below for the use of this option. If it is not set when quota_warn_threshold is set, it defaults to

quota_warn_message = "\
  To: $local_part@$domain\n\
  Subject: Your mailbox\n\n\
  This message is automatically created \
  by mail delivery software.\n\n\
  The size of your mailbox has exceeded \
  a warning threshold that is\n\
  set by the system administrator.\n"

This option is expanded in the same way as quota (see above). If the resulting value is greater than zero, and delivery of the message causes the size of the file or total space in the directory tree to cross the given threshold, a warning message is sent. If quota is also set, the threshold may be specified as a percentage of it by following the value with a percent sign. For example:

quota = 10M
quota_warn_threshold = 75%

If quota is not set, a setting of quota_warn_threshold that ends with a percent sign is ignored.

The warning message itself is specified by the quota_warn_message option, and it must start with a To: header line containing the recipient(s) of the warning message. These do not necessarily have to include the recipient(s) of the original message. A Subject: line should also normally be supplied. You can include any other header lines that you want. If you do not include a From: line, the default is:

From: Mail Delivery System <mailer-daemon@$qualify_domain_sender>

If you supply a Reply-To: line, it overrides the global errors_reply_to option.

The quota option does not have to be set in order to use this option; they are independent of one another except when the threshold is specified as a percentage.

If this option is set true, appendfile writes messages in "batch SMTP" format, with the envelope sender and recipient(s) included as SMTP commands. If you want to include a leading HELO command with such messages, you can do so by setting the message_prefix option. See section Outgoing batched SMTP for details of batch SMTP.

This option causes lines to be terminated with the two-character CRLF sequence (carriage return, linefeed) instead of just a linefeed character. In the case of batched SMTP, the byte sequence written to the file is then an exact image of what would be sent down a real SMTP connection.

The contents of the message_prefix and message_suffix options are written verbatim, so must contain their own carriage return characters if these are needed. In cases where these options have non-empty defaults, the values end with a single linefeed, so they must be changed to end with `\r\n' if use_crlf is set.

This option controls the use of the fcntl() function to lock a file for exclusive use when a message is being appended. It is set by default unless use_flock_lock is set. Otherwise, it should be turned off only if you know that all your MUAs use lock file locking. When both use_fcntl_lock and use_flock_lock are unset, use_lockfile must be set.

This option is provided to support the use of flock() for file locking, for the few situations where it is needed. Most modern operating systems support fcntl() and lockf() locking, and these two functions interwork with each other. Exim uses fcntl() locking by default.

This option is required only if you are using an operating system where flock() is used by programs that access mailboxes (typically MUAs), and where flock() does not correctly interwork with fcntl(). You can use both fcntl() and flock() locking simultaneously if you want.

Not all operating systems provide flock(). Some versions of Solaris do not have it (and some, I think, provide a not quite right version built on top of lockf()). If the OS does not have flock(), Exim will be built without the ability to use it, and any attempt to do so will cause a configuration error.

Warning: flock() locks do not work on NFS files (unless flock() is just being mapped onto fcntl() by the OS).

If this option is turned off, Exim does not attempt to create a lock file when appending to a mailbox file. In this situation, the only locking is by fcntl(). You should only turn use_lockfile off if you are absolutely sure that every MUA that is ever going to look at your users' mailboxes uses fcntl() rather than a lock file, and even then only when you are not delivering over NFS from more than one host.

In order to append to an NFS file safely from more than one host, it is necessary to take out a lock before opening the file, and the lock file achieves this. Otherwise, even with fcntl() locking, there is a risk of file corruption.

The use_lockfile option is set by default unless use_mbx_lock is set. It is not possible to turn both use_lockfile and use_fcntl_lock off, except when mbx_format is set.

This option is available only if Exim has been compiled with SUPPORT_MBX set in `Local/Makefile'. Setting the option specifies that special MBX locking rules be used. It is set by default if mbx_format is set and none of the locking options are mentioned in the configuration. The locking rules are the same as are used by the c-client library that underlies Pine and the IMAP4 and POP daemons that come with it (see the discussion below). The rules allow for shared access to the mailbox. However, this kind of locking does not work when the mailbox is NFS mounted.

You can set use_mbx_lock with either (or both) of use_fcntl_lock and use_flock_lock to control what kind of locking is used in implementing the MBX locking rules. The default is to use fcntl() if use_mbx_lock is set without use_fcntl_lock or use_flock_lock.

26.3 Operational details for appending

Before appending to a file, the following preparations are made:

• If the name of the file is `/dev/null', no action is taken, and a success return is given.
• If any directories on the file's path are missing, Exim creates them if the create_directory option is set. A created directory's mode is given by the directory_mode option.
• If file_format is set, the format of an existing file is checked. If this indicates that a different transport should be used, control is passed to that transport.
• If use_lockfile is set, a lock file is built in a way that will work reliably over NFS, as follows:
  1. Create a "hitching post" file whose name is that of the lock file with the current time, primary host name, and process id added, by opening for writing as a new file. If this fails with an access error, delivery is deferred.
  2. Close the hitching post file, and hard link it to the lock file name.
  3. If the call to link() succeeds, creation of the lock file has succeeded. Unlink the hitching post name.
  4. Otherwise, use stat() to get information about the hitching post file, and then unlink hitching post name. If the number of links is exactly two, creation of the lock file succeeded but something (for example, an NFS server crash and restart) caused this fact not to be communicated to the link() call.
  5. If creation of the lock file failed, wait for lock_interval and try again, up to lock_retries times. However, since any program that writes to a mailbox should complete its task very quickly, it is reasonable to time out old lock files that are normally the result of user agent and system crashes. If an existing lock file is older than lockfile_timeout Exim attempts to unlink it before trying again.
• A call is made to lstat() to discover whether the main file exists, and if so, what its characteristics are. If lstat() fails for any reason other than non-existence, delivery is deferred.
• If the file does exist and is a symbolic link, delivery is deferred, unless the allow_symlink option is set, in which case the ownership of the link is checked, and then stat() is called to find out about the real file, which is then subjected to the checks below. The check on the top-level link ownership prevents one user creating a link for another's mailbox in a sticky directory, though allowing symbolic links in this case is definitely not a good idea. If there is a chain of symbolic links, the intermediate ones are not checked.
• If the file already exists but is not a regular file, or if the file's owner and group (if the group is being checked - see check_group above) are different from the user and group under which the delivery is running, delivery is deferred.
• If the file's permissions are more generous than specified, they are reduced. If they are insufficient, delivery is deferred, unless mode_fail_narrower is set false, in which case the delivery is tried using the existing permissions.
• The file's inode number is saved, and the file is then opened for appending. If this fails because the file has vanished, appendfile behaves as if it hadn't existed (see below). For any other failures, delivery is deferred.
• If the file is opened successfully, check that the inode number hasn't changed, that it is still a regular file, and that the owner and permissions have not changed. If anything is wrong, defer delivery and freeze the message.
• If the file did not exist originally, defer delivery if the file_must_exist option is set. Otherwise, check that the file is being created in a permitted directory if the create_file option is set (deferring on failure), and then open for writing as a new file, with the O_EXCL and O_CREAT options, except when dealing with a symbolic link (the allow_symlink option must be set). In this case, which can happen if the link points to a non-existent file, the file is opened for writing using O_CREAT but not O_EXCL, because that prevents link following.
• If opening fails because the file exists, obey the tests given above for existing files. However, to avoid looping in a situation where the file is being continuously created and destroyed, the exists/not-exists loop is broken after 10 repetitions, and the message is then frozen.
• If opening fails with any other error, defer delivery.
• Once the file is open, unless both use_fcntl_lock and use_flock_lock are false, it is locked using fcntl() or flock() or both. If use_mbx_lock is false, an exclusive lock is requested in each case. However, if use_mbx_lock is true, Exim takes out a shared lock on the open file, and an exclusive lock on the file whose name is

  /tmp/.<device-number>.<inode-number>

  using the device and inode numbers of the open mailbox file, in accordance with the MBX locking rules. This file is created with a mode that is specified by the lockfile_mode option.

  If Exim fails to lock the file, there are two possible courses of action, depending on the value of the locking timeout. This is obtained from lock_fcntl_timeout or lock_flock_timeout, as appropriate.

  If the timeout value is zero, the file is closed, Exim waits for lock_interval, and then goes back and re-opens the file as above and tries to lock it again. This happens up to lock_retries times, after which the delivery is deferred.

  If the timeout has a value greater than zero, blocking calls to fcntl() or flock() are used (with the given timeout), so there has already been some waiting involved by the time locking fails. Nevertheless, Exim does not give up immediately. It retries up to

  (lock_retries * lock_interval) / <timeout>

  times (rounded up).

At the end of delivery, Exim closes the file (which releases the fcntl() and/or flock() locks) and then deletes the lock file if one was created.

26.4 Operational details for delivery to a new file

When the directory option is set instead of file, each message is delivered into a newly-created file or set of files. When appendfile is activated directly from a redirect router, neither file nor directory is normally set, because the path for delivery is supplied by the router. (See for example, the address_file transport in the default configuration.) In this case, delivery is to a new file if either the path name ends in `/', or the maildir_format or mailstore_format option is set.

No locking is required while writing the message to a new file, so the various locking options of the transport are ignored. The "From" line that by default separates messages in a single file is not normally needed, nor is the escaping of message lines that start with "From", and there is no need to ensure a newline at the end of each message. Consequently, the default values for check_string, message_prefix, and message_suffix are all unset when any of directory, maildir_format, or mailstore_format is set.

If Exim is required to check a quota setting, it adds up the sizes of all the files in the delivery directory by default. However, you can specify a different directory by setting quota_directory. Also, for maildir deliveries (see below) the `maildirfolder' convention is honoured.

There are three different ways in which delivery to individual files can be done, controlled by the settings of the maildir_format and mailstore_format options. Note that code to support maildir or mailstore formats is not included in the binary unless SUPPORT_MAILDIR or SUPPORT_MAILSTORE, respectively, is set in `Local/Makefile'.

In all three cases an attempt is made to create the directory and any necessary sub-directories if they do not exist, provided that the create_directory option is set (the default). The location of a created directory can be constrained by setting create_file. A created directory's mode is given by the directory_mode option. If creation fails, or if the create_directory option is not set when creation is required, delivery is deferred.

26.5 Maildir delivery

If the maildir_format option is true, Exim delivers each message by writing it to a file whose name is `tmp/<stime>.H<mtime>P<pid>.<host>' in the directory that is defined by the directory option (the "delivery directory"). If the delivery is successful, the file is renamed into the `new' subdirectory.

In the file name, <stime> is the current time of day in seconds, and <mtime> is the microsecond fraction of the time. After a maildir delivery, Exim checks that the time-of-day clock has moved on by at least one microsecond before terminating the delivery process. This guarantees uniqueness for the file name. However, as a precaution, Exim calls stat() for the file before opening it. If any response other than ENOENT (does not exist) is given, Exim waits 2 seconds and tries again, up to maildir_retries times.

Before Exim carries out a maildir delivery, it ensures that subdirectories called `new', `cur', and `tmp' exist in the delivery directory. If they do not exist, Exim tries to create them and any superior directories in their path, subject to the create_directory and create_file options. If the maildirfolder_create_regex option is set, and the regular expression it contains matches the delivery directory, Exim also ensures that a file called `maildirfolder' exists in the delivery directory. If a missing directory or `maildirfolder' file cannot be created, delivery is deferred.

These features make it possible to use Exim to create all the necessary files and directories in a maildir mailbox, including subdirectories for maildir++ folders. Consider this example:

maildir_format = true
directory = /var/mail/$local_part\
           ${if eq{$local_part_suffix}{}{}\
           {/.${substr_1:$local_part_suffix}}}
maildirfolder_create_regex = /\.[^/]+$

If $local_part_suffix is empty (there was no suffix for the local part), delivery is into a toplevel maildir with a name like `/var/mail/pimbo' (for the user called pimbo). The pattern in maildirfolder_create_regex does not match this name, so Exim will not look for or create the file `/var/mail/pimbo/maildirfolder', though it will create `/var/mail/pimbo/{cur,new,tmp}' if necessary.

However, if $local_part_suffix contains `-eximusers' (for example), delivery is into the maildir++ folder `/var/mail/pimbo/.eximusers', which does match maildirfolder_create_regex. In this case, Exim will create `/var/mail/pimbo/.eximusers/maildirfolder' as well as the three maildir directories `/var/mail/pimbo/.eximusers/{cur,new,tmp}'.

Warning: Take care when setting maildirfolder_create_regex that it does not inadvertently match the toplevel maildir directory, because a `maildirfolder' file at top level would completely break quota calculations.

If Exim is required to check a quota setting before a maildir delivery, and quota_directory is not set, it looks for a file called `maildirfolder' in the maildir directory (alongside `new', `cur', `tmp'). If this exists, Exim assumes the directory is a maildir++ folder directory, which is one level down from the user's top level mailbox directory. This causes it to start at the parent directory instead of the current directory when calculating the amount of space used.

One problem with delivering into a multi-file mailbox is that it is computationally expensive to compute the size of the mailbox for quota checking. Various approaches have been taken to reduce the amount of work needed. The next two sections describe two of them. A third alternative is to use some external process for maintaining the size data, and use the expansion of the mailbox_size option as a way of importing it into Exim.

26.6 Using tags to record message sizes

If maildir_tag is set, the string is expanded for each delivery. When the maildir file is renamed into the `new' sub-directory, the tag is added to its name. However, if adding the tag takes the length of the name to the point where the test stat() call fails with ENAMETOOLONG, the tag is dropped and the maildir file is created with no tag.

Tags can be used to encode the size of files in their names; see quota_size_regex above for an example. The expansion of maildir_tag happens after the message has been written. The value of the $message_size variable is set to the number of bytes actually written. If the expansion is forced to fail, the tag is ignored, but a non-forced failure causes delivery to be deferred. The expanded tag may contain any printing characters except "/". Non-printing characters in the string are ignored; if the resulting string is empty, it is ignored. If it starts with an alphanumeric character, a leading colon is inserted.

26.7 Using a maildirsize file

If maildir_use_size_file is true, Exim implements the maildir++ rules for storing quota and message size information in a file called `maildirsize' within the toplevel maildir directory. If this file does not exist, Exim creates it, setting the quota from the quota option of the transport. If the maildir directory itself does not exist, it is created before any attempt to write a `maildirsize' file.

The `maildirsize' file is used to hold information about the sizes of messages in the maildir, thus speeding up quota calculations. The quota value in the file is just a cache; if the quota is changed in the transport, the new value overrides the cached value when the next message is delivered. The cache is maintained for the benefit of other programs that access the maildir and need to know the quota.

If the quota option in the transport is unset or zero, the `maildirsize' file is maintained (with a zero quota setting), but no quota is imposed.

A regular expression is available for controlling which directories in the maildir participate in quota calculations when a `maildirsizefile' is in use. See the description of the maildir_quota_directory_regex option above for details.

26.8 Mailstore delivery

If the mailstore_format option is true, each message is written as two files in the given directory. A unique base name is constructed from the message id and the current delivery process, and the files that are written use this base name plus the suffixes `.env' and `.msg'. The `.env' file contains the message's envelope, and the `.msg' file contains the message itself. The base name is placed in the variable $mailstore_basename.

During delivery, the envelope is first written to a file with the suffix `.tmp'. The `.msg' file is then written, and when it is complete, the `.tmp' file is renamed as the `.env' file. Programs that access messages in mailstore format should wait for the presence of both a `.msg' and a `.env' file before accessing either of them. An alternative approach is to wait for the absence of a `.tmp' file.

The envelope file starts with any text defined by the mailstore_prefix option, expanded and terminated by a newline if there isn't one. Then follows the sender address on one line, then all the recipient addresses, one per line. There can be more than one recipient only if the batch_max option is set greater than one. Finally, mailstore_suffix is expanded and the result appended to the file, followed by a newline if it does not end with one.

If expansion of mailstore_prefix or mailstore_suffix ends with a forced failure, it is ignored. Other expansion errors are treated as serious configuration errors, and delivery is deferred. The variable $mailstore_basename is available for use during these expansions.

26.9 Non-special new file delivery

If neither maildir_format nor mailstore_format is set, a single new file is created directly in the named directory. For example, when delivering messages into files in batched SMTP format for later delivery to some host (see section Outgoing batched SMTP), a setting such as

directory = /var/bsmtp/$host

might be used. A message is written to a file with a temporary name, which is then renamed when the delivery is complete. The final name is obtained by expanding the contents of the directory_file option.

This document was generated on July, 10 2007 using texi2html 1.76.