SME FetchMail

Subject

SME is a very nice mail server, but we sometime need external mailboxes. In this case, retrieving mails from theses mailboxes can be a lot of work. Moreover, if you configure a lot of P.C. on your network to retrieve a bunch of externals mailboxes, the configuration of each computer becomes complicated.
It could be easier (and maybe safer) to let our SME server drop all mails and store them in each user box(es), allowing a centralized and standardized mail configuration

Here comes fetchmail in the process. Fetchmail is intended just to do that : at regular time, retrieve a lot of externals mailboxes, and throw the contents to internals mailboxes.
You should note, however, that fetchmail only retrieve mail from PO3 an IMAP4 server. You cannot use it to retrieve mail from a webmail server.

Installation

Fetchmail itself is already installed on your SME Server : it is used by the ETRN and multidrop mode of the e-mail configuration (See the SME documentation for more information)

Here we speak about an add-on for SME, who is intended to retrieve several externals mailboxes for several internals accounts

To my knowledge, there are two versions of fetchmail for SME.

• sme-fetchmail-1.0.2 is compatible with SME up to 5.6, but encounter some minors troubles on SME 6.0 and upper. This page don't speak about this version but if you need it, you can have a look on the great Swerts-Knudsen.dk site.
• smeserver-fetchmail-1.3.4 (and upper) is a complete rework, and works only on SME 6.0 and upper. We speak here about this version.

prerequisites

To install smeserver-fetchmail, you need :

• SME Server version 6.0 or newer
• perl-CGI-FormMagick version >= 0.89-08 (skip to the next step if your SME server if over version 6.0 !)

'Academic' way

rpm -q perl-CGI-FormMagick
perl-CGI-FormMagick-0.89-08

If your level is below this one, you need to upgrade FormMagick.

'Funny' way

Just start the installation of smeserver-fetchmail. If your system don't reach the FormMagick desired level, it will tell you !

FormMagick Upgrade

FormMagick-0.89-08 is available from all majors SME mirrors, but is still (in June 2004) in the 'devel' subdir.
You can install it directly from my personal site, like this :

rpm -Uvh http://www.schirrms.net/files/perl-CGI-FormMagick-0.89-08.noarch.rpm
Retrieving http://www.schirrms.net/files/perl-CGI-FormMagick-0.89-08.noarch.rpm
Preparing...                ########################################### [100%]
   1:perl-CGI-FormMagick    ########################################### [100%]

smeserver-fetchmail RPM installation

Installation is easy. just one RPM to roll :

rpm -Uvh http://www.schirrms.net/files/smeserver-fetchmail-utf8-1.3.5-02.noarch.rpm
Retrieving http://www.schirrms.net/files/smeserver-fetchmail-1.3.5-02.noarch.rpm
Preparing...                ########################################### [100%]
   1:smeserver-fetchmail-utf########################################### [100%]
Rebuilding Web Server Manager Left Panel Cache ... Can take up to a minute.
Done.
Setting defaults values in SME configuration database, if needed. Don't change any existing configuration.
Migrating existing database domains
Migrating existing database spamassassin
Migrating existing database configuration
Migrating existing database hosts
Migrating existing database yum_repositories
Migrating existing database yum_available
Migrating existing database backups
Migrating existing database networks
Migrating existing database mailpatterns
Migrating existing database accounts
Migrating existing database yum_updates
Migrating existing database yum_installed
Done.
Installation finished.

For your information, the source RPM (totally unneeded for installation) is also available

In case of an upgrade

Normally, if you upgrade from sme-fetchmail-1.0.2, everything should be OK. At least, we did numerous tests for that !

As a complementary safety measure, you can backup these two files, who contains the whole fetchmail configuration :
/home/e-smith/configuration or /home/e-smith/db/configuration
/home/e-smith/accounts or /home/e-smith/db/accounts

Two points are to change manually :

• If you enabled the multidrop mode or the ETRN mode to enable fetchmail, this is no more needed. You can set back to 'standard' mode.
• If you only installed sme-upgppp-1.0-1.noarch.rpm for fetchmail, you no more need it. you can uninstall it ( rpm -ev sme-upgppp ).

Configuration

In the server manager, part 'Collaboration' you should find a 'Mails Retrieval' option.

This let you access on the main page of the configuration.

Main screen

You can see a screenshot here.

Here, you can set some generals options :

- A transparent SMTP proxy has been added, to force all outgoing
SMTP connections to go through the SME server. This feature
enhances security with respect to viruses and worms that use SMTP
to propagate themselves.

Note : you cannot change the time schedule of fetchmail here. This is because fetchmail is still sharing some parts with the standard SME Fetchmail system.
If the standard schedule (every 15 minutes) doesn't suit your needs, in server manager, go on Configuration -> E-Mail and choose new schedule rules (at the bottom of the page).
But there is a trick : If your server E-mail retrieval mode (at the top of the same page) is set as Standard (SMTP), you cannot save the change !
In this case just select ETRN (SMTP with client request), do your schedule change, and click 'Save'.
Then go again in the page, select back Standard (SMTP), and save again.

You can also select any SME user, to create a new fetchmail ruleset, or to modify or remove the existing ruleset.

User screen 1

Then you go to the first user page (screenshot)
This screen let you set recipients rules for the mail retrieved by fetchmail. In most case, you wont change anything here.

Here, four options :

Just below, you have a list of internals accounts. Just check the corresponding case to have all incoming mails send also to this(these) account(s).

You can use any combination of local copy (that is "ghost account = no") copy to internal(s) user(s) and copy to an external mailbox. But you need at least one of them, and fetchmail will tell you if you forgot it !

User screen 2

In this screen, you set the information for the mailboxes to fetch. A screenshot is here.

You can retrieve many mailboxes for a single account (I think there is no limit, but if you find one, let me know)

You can create, modify and delete externals mailboxes configuration.
For each external mailbox to fetch, you must give four informations :

And finally, you'll find a [Validate changes] button. Don't forget it ! Your user modifications are only stored when you click on it !

Configuration tuning

# to see the current 'fetchhost' value :
/sbin/e-smith/db configuration getprop FetchMails fetchhost

# to set a value (no validity chek, be careful)
/sbin/e-smith/db configuration setprop FetchMails fetchhost smtp.from.mycorp.com

# check the new value
/sbin/e-smith/db configuration getprop FetchMails fetchhost
smtp.from.mycorp.com

# delete the value
/sbin/e-smith/db configuration delprop FetchMails fetchhost

#IMPORTANT ! activate the changes !
/sbin/e-smith/expand-template /etc/fetchmail

# to see the current 'timeout' value :
/sbin/e-smith/db configuration getprop FetchMails timeout
60

# to set a value (no validity chek, be careful) expressed in seconds
/sbin/e-smith/db configuration setprop FetchMails timeout 30

# check the new value
/sbin/e-smith/db configuration getprop FetchMails timeout
30

#IMPORTANT ! activate the changes !
/sbin/e-smith/expand-template /etc/fetchmail

Notes for peoples upgrading from sme-fetchmail

Here again, for most cases, upgrade should work fine. However, even if the general 'look & feel' is fairly the same a lot of underlying work have been done. So you could have smalls changes during operations.

• The spy option is no more present. But if you have set this option before, you should note that now, in the 'mail copy-to' section, the 'admin' case is checked.
• There's no more an 'What action with the mails to forward' option : if you set an external E-Mail mail is send to this external E-mail. If the case is blank, mail follow others rules only. Remember : you can use any combination of mail distribution.
• You can now set fetchmail rules for the admin account.

About smeserver-fetchmail and SME starting at version 7.1

SME 7.0 was the first SME version with an embedded spamfilter and antivirus program (and that's great). But on alot of small systems, the time to accept a mail was toooo long. The SME Core Team choose to suppress the spam filter for mail coming from the LAN. No luck, for SME, mails coming via fetchmails were seen as local. On SME 7.1, I didn't see a way to go trough this change and incoming mails weren't checked. Lucky me, the trouble was the same for mails coming via ETRN or Multidrop, wich are standard functions of SME. So the Core Team put a solution in SME starting at version 7.1.1. I 'just' had to create a new version of smeserver-fetchmail using this new way for incoming mail.

It would probably be a way to create a smeserver-fetchmail package working on all SME versions, but the overwork it would add was to much for me. So we have two versions in the wild : 1.3.4-10 for system up to SME 7.0 and the current version (1.3.5-00 at the time of writing) for SME from 7.1.1.

And what about SME 7.1.0 ? I have to say that this version is not officially supported. Of course, you can install smeserver-fetchmail-1.3.4-10 on it, but this will work without antispam filter, witch seems a strange idea today

This is not only bad, in fact : the new version of smeserver-fetchmail can use new features of fetchmail. So I was able to add support for POP3 and IMAP via SSL ("About time" would you say :-) )

About smeserver-fetchmail and SME 7.3 with the April 2008 Update

Starting with the update of April 2008, SME did a major change (even if hard to see) : the system charset went from ISO-8859-1 to UTF-8 ! What does it mean ? Computers store any information as a succession of bits (that is O or 1). This is not really 'user friendly' :-) To be able to store text in a computer file, we need to associate bit succession with human character. So, in most small systems the letter A is equal to 65 (or 0100 0001) and the number one is aqual to 49 (or 0011 0001). For non american character, there are many coding tables. A fairly recent coding table UTF-8, is for now the most commonly accepted table. This table permit to store most known symbols (Greek, Chinese...). But the change is big, and I don't see a way to create a package runing in the two charset, so I created a specific package for the most updated SME server and another, neeeded until all SME server be updated (can takes years !).

Wish List : Ideas to improve/extend this contribution

smeserver-fetchmail seems to have many users, and some of them gave me some greats ideas to improve or extend the contribution. I'll try to settle all theses ideas here. Don't hesitate to send me others ideas (by mail), I'll try to put them here also.
But I have very little free time, and I cannot say that these ideas will be included in the contribution !
If YOU think that one of theses ideas really need to be included, don't hesitate to do the work ! You're welcome !
The ideas are here without any order.

Contrib authors

The initial sme-fetchmail was written by Vincent Filali (his site). The whole fetchmail concept is from him.

The current version, smeserver-fetchmail is a complete interface rework, but the base engine is still the same. This work and the documentation was done by me, Pascal Schirrmann (my site), under supervision of Vincent.

Change Log

copyleft Schirrms Studio 2002-2004

The whole website www.schirrms.net is under GFPL license