|Tested on||SME 6.0, 6.0.1, 6.5RC1, 7.0, 7.1, 7.2, 7.3 and 7.3 utf-8|
SME is a very nice mail server, but we sometime need external mailboxes. In this case, retrieving
mails from theses mailboxes can be a lot of work. Moreover, if you configure a lot of P.C. on
your network to retrieve a bunch of externals mailboxes, the configuration of each computer
It could be easier (and maybe safer) to let our SME server drop all mails and store them in each user box(es), allowing a centralized and standardized mail configuration
Here comes fetchmail in the process. Fetchmail is intended just to do that :
at regular time,
retrieve a lot of externals mailboxes, and throw the contents to internals mailboxes.
You should note, however, that fetchmail only retrieve mail from PO3 an IMAP4 server. You cannot use it to retrieve mail from a webmail server.
Fetchmail itself is already installed on your SME Server : it is used by the ETRN and multidrop mode of the e-mail configuration (See the SME documentation for more information)
Here we speak about an add-on for SME, who is intended to retrieve several externals mailboxes for several internals accounts
To my knowledge, there are two versions of fetchmail for SME.
To install smeserver-fetchmail, you need :
rpm -q perl-CGI-FormMagick perl-CGI-FormMagick-0.89-08
If your level is below this one, you need to upgrade FormMagick.
Just start the installation of smeserver-fetchmail. If your system don't reach the FormMagick desired level, it will tell you !
FormMagick-0.89-08 is available from all majors SME mirrors, but is still (in June 2004) in the 'devel' subdir.
You can install it directly from my personal site, like this :
rpm -Uvh http://www.schirrms.net/files/perl-CGI-FormMagick-0.89-08.noarch.rpm Retrieving http://www.schirrms.net/files/perl-CGI-FormMagick-0.89-08.noarch.rpm Preparing... ########################################### [100%] 1:perl-CGI-FormMagick ########################################### [100%]
Installation is easy. just one RPM to roll :
rpm -Uvh http://www.schirrms.net/files/smeserver-fetchmail-utf8-1.3.5-02.noarch.rpm Retrieving http://www.schirrms.net/files/smeserver-fetchmail-1.3.5-02.noarch.rpm Preparing... ########################################### [100%] 1:smeserver-fetchmail-utf########################################### [100%] Rebuilding Web Server Manager Left Panel Cache ... Can take up to a minute. Done. Setting defaults values in SME configuration database, if needed. Don't change any existing configuration. Migrating existing database domains Migrating existing database spamassassin Migrating existing database configuration Migrating existing database hosts Migrating existing database yum_repositories Migrating existing database yum_available Migrating existing database backups Migrating existing database networks Migrating existing database mailpatterns Migrating existing database accounts Migrating existing database yum_updates Migrating existing database yum_installed Done. Installation finished.
For your information, the source RPM (totally unneeded for installation) is also available
Normally, if you upgrade from sme-fetchmail-1.0.2, everything should be OK. At least, we did numerous tests for that !
As a complementary safety measure, you can backup these two files, who contains the whole fetchmail configuration :
/home/e-smith/configuration or /home/e-smith/db/configuration
/home/e-smith/accounts or /home/e-smith/db/accounts
Two points are to change manually :
In the server manager, part 'Collaboration' you should find a 'Mails Retrieval' option.
This let you access on the main page of the configuration.
You can see a screenshot here.
Here, you can set some generals options :
|Enable/Disable Fetchmail||Just stop or start Fetchmail, without any change to the configuration. The following filters are still working !|
|Enable/Disable External POP and IMAP server Access||If you disable that, your internals stations cannot more access externals POP3 and/or IMAP4 servers. Only the SME server can do that. In others words : Fetchmail is mandatory.|
|Enable/Disable access to external News servers||If you select to disable access to news (NNTP) servers, all stations on your LAN loose access to any external News servers. To use if you have a local reflector or if you think that news are not needed.|
|Enable/Disable access to external SMTP servers||If you disable access to external SMTP servers, all stations on your LAN need to use the SME SMTP services to send E-Mail. Note that this has only effect in conjunction with the next parameter.|
|Use the SME SMTP Proxy (yes/no)||SME SMTP Proxy is a new feature of SME 6. From the SME documentation :
- A transparent SMTP proxy has been added, to force all outgoing SMTP connections to go through the SME server. This feature enhances security with respect to viruses and worms that use SMTP to propagate themselves.The SME SMTP Option is enable by default. This means that every outgoing SMTP connection is in fact 'catched' by SME, and only the SME server does connection to external SMTP server. In most cases, this option is wise. If you let this option on (green light ;-) ), the the upper option has no effect (in fact, stations on your LAN can only connect to SME). If you choose (or need) to deactivate the SME SMTP Proxy, then the upper option becomes valid.
Note : you cannot change the time schedule of fetchmail here. This is because fetchmail is still sharing some
parts with the standard SME Fetchmail system.
If the standard schedule (every 15 minutes) doesn't suit your needs, in server manager, go on Configuration -> E-Mail and choose new schedule rules (at the bottom of the page).
But there is a trick : If your server E-mail retrieval mode (at the top of the same page) is set as Standard (SMTP), you cannot save the change !
In this case just select ETRN (SMTP with client request), do your schedule change, and click 'Save'.
Then go again in the page, select back Standard (SMTP), and save again.
You can also select any SME user, to create a new fetchmail ruleset, or to modify or remove the existing ruleset.
Then you go to the first user page (screenshot)
This screen let you set recipients rules for the mail retrieved by fetchmail. In most case, you wont change anything here.
Here, four options :
|Is it a 'ghost' account (only to redistribute mails) ?||If you choose yes, this account will search externals mail, but will only distribute them on another internals mailboxes and/or externals mails address.|
|Do you want to debug this account (fetchmail actions are logged in /var/log/maillog) ?||Really only a debug option, as it is very verbose !|
|What do you want to do with E-Mails on externals servers ?||Three options :
|Email address(es) to witch forward mails ( format : email@example.com or firstname.lastname@example.org : email@example.com )||If you want that every mail is send to an external e-mail, just set it here.
You can set here more than one external e-mail address. Just set a colon between each email addresses.
Just below, you have a list of internals accounts. Just check the corresponding case to have all incoming mails send also to this(these) account(s).
You can use any combination of local copy (that is "ghost account = no") copy to internal(s) user(s) and copy to an external mailbox. But you need at least one of them, and fetchmail will tell you if you forgot it !
In this screen, you set the information for the mailboxes to fetch. A screenshot is here.
You can retrieve many mailboxes for a single account (I think there is no limit, but if you find one, let me know)
You can create, modify and delete externals mailboxes configuration.
For each external mailbox to fetch, you must give four informations :
|Name of the External Mail Server||Internet style name, like imap.myisp.net or pop3.mailservice.com|
|Protocol to use when reading mails||Starting with version 1.3.5, four choices : POP3, IMAP4, POP3 via SSL (experimental) and IMAP4 via SSL (very experimental !)|
|The user account name on the mail server||This is the login for the account on the mail system.|
|The user account password on the mail server||This is the associated password to gain access on the mail system. Please note : the password can be seen clearly on this screen and is almost so clear to read in the config file. This file can only be read by a system administrator. If this is an issue for you, you should maybe try another way. But remember : Using POP3 or IMAP4, your password are traveling totally clearly on Internet.|
And finally, you'll find a [Validate changes] button. Don't forget it ! Your user modifications are only stored when you click on it !
smeserver-fetchmail-1.3.4-06 arrives with some tiny configuration changes, for the mail redistribution logic.
fechtmail retrieves mails using POP3 or IMAP4 protocol, following your configuration choices, and then deliver these mails by using the SMTP protocol.
Older smeserver-fetchmail version ever deliver the mail to the local SMTP server.
Now, the logic is the following :
In some cases, smeserver-fetchmail cannot guess alone how to deliver the retrieved mail. So I have added a configuration field that you can use to set a destination server (the server who own the final mailboxes).Probably, in the future, you'll be able to feed this field from the server-manager fetchmail panel. But for now, if you need this configuration change, you can do it so :
# to see the current 'fetchhost' value : /sbin/e-smith/db configuration getprop FetchMails fetchhost # to set a value (no validity chek, be careful) /sbin/e-smith/db configuration setprop FetchMails fetchhost smtp.from.mycorp.com # check the new value /sbin/e-smith/db configuration getprop FetchMails fetchhost smtp.from.mycorp.com # delete the value /sbin/e-smith/db configuration delprop FetchMails fetchhost #IMPORTANT ! activate the changes ! /sbin/e-smith/expand-template /etc/fetchmailAs the /etc/fetchmail file is reread at each fetchmail collect, you don't need to restart any services to activate the change.
Starting with version 1.3.5, a 'timeout' parameter has been added. Before, if the remote server didn't answer to a fetchmail query,fetchmail did stop after a timeout hardcoded at compilation time ! Now the timeout is by default of 60 seconds. This parameter is the same for all fetchmail accounts. If this 60 seconds value doesn't suit your needs, you can change it :
# to see the current 'timeout' value : /sbin/e-smith/db configuration getprop FetchMails timeout 60 # to set a value (no validity chek, be careful) expressed in seconds /sbin/e-smith/db configuration setprop FetchMails timeout 30 # check the new value /sbin/e-smith/db configuration getprop FetchMails timeout 30 #IMPORTANT ! activate the changes ! /sbin/e-smith/expand-template /etc/fetchmail
Here again, for most cases, upgrade should work fine. However, even if the general 'look & feel' is fairly the same a lot of underlying work have been done. So you could have smalls changes during opérations.
SME 7.0 was the first SME version with an embedded spamfilter and antivirus program (and that's great). But on alot of small systems, the time to accept a mail was toooo long. The SME Core Team choose to suppress the spam filter for mail coming from the LAN. No luck, for SME, mails coming via fetchmails were seen as local. On SME 7.1, I didn't see a way to go trough this change and incoming mails weren't checked. Lucky me, the trouble was the same for mails coming via ETRN or Multidrop, wich are standard functions of SME. So the Core Team put a solution in SME starting at version 7.1.1. I 'just' had to create a new version of smeserver-fetchmail using this new way for incoming mail.
It would probably be a way to create a smeserver-fetchmail package working on all SME versions, but the overwork it would add was to much for me. So we have two versions in the wild : 1.3.4-10 for system up to SME 7.0 and the current version (1.3.5-00 at the time of writing) for SME from 7.1.1.
And what about SME 7.1.0 ? I have to say that this version is not officially supported. Of course, you can install smeserver-fetchmail-1.3.4-10 on it, but this will work without antispam filter, witch seems a strange idea today
This is not only bad, in fact : the new version of smeserver-fetchmail can use new features of fetchmail. So I was able to add support for POP3 and IMAP via SSL ("About time" would you say :-) )
Starting with the update of April 2008, SME did a major change (even if hard to see) : the system charset went from ISO-8859-1 to UTF-8 ! What does it mean ? Computers store any information as a succession of bits (that is O or 1). This is not really 'user friendly' :-) To be able to store text in a computer file, we need to associate bit succession with human character. So, in most small systems the letter A is equal to 65 (or 0100 0001) and the number one is aqual to 49 (or 0011 0001). For non american character, there are many coding tables. A fairly recent coding table UTF-8, is for now the most commonly accepted table. This table permit to store most known symbols (Greek, Chinese...). But the change is big, and I don't see a way to create a package runing in the two charset, so I created a specific package for the most updated SME server and another, neeeded until all SME server be updated (can takes years !).
smeserver-fetchmail seems to have many users, and some of them gave me some greats ideas to improve or extend the contribution.
I'll try to settle all theses ideas here. Don't hesitate to send me others ideas (by mail),
I'll try to put them here also.
But I have very little free time, and I cannot say that these ideas will be included in the contribution !
If YOU think that one of theses ideas really need to be included, don't hesitate to do the work ! You're welcome !
The ideas are here without any order.
|Fetchmail for groups||For now, if you want that a mail 'fetched' is send to more than one user, you'll have to check every recipients by hand. it could be easier to use SME group as recipient : configuration easier, and the change in a group would be automatically active on Fetchmail|
|Fetchmail configuration backup||If you have to migrate a fetchmail configuration from one SME server to another, you have to recreate the whole configuration. It would be marvelous to have an export/import tool, like the great lazy-admin-tools.|
|External mail account 'enable/disable' checkbox||Most useful in test and debug, the idea is to have in from of each POP/IMAP account, a 'enable' checkbox. This would permit to temporary disable an account without needing to remove and recreate it.|
|Secure pop3 retrieval||Some mail services (GMail from Google for instance) can only be accessed via secure POP3 (or maybe secure IMAP). The fetchmail binary send with SME doesn't have the ssl extension embedded. So it's not only a configuration option to add...This feature is now supported since version 1.3.5-00 (yep !)|
|Custom port number for the collected services||TCP/IP gives for each well known service a port number. For example, 110 is for POP3 and 143 is for IMAP4. But some providers use non standard port number. It could be useful to be able to set these non standard port numbers.|
|IMAP4 folder selection||By default, fetchmail retrieve mails in the distant INBOX folder. But it's possible to ask fetchmail to retrieve mails in other folder(s).|
|Webmail fetching||How to fetch webmail servers. It seems to be possible, but it's maybe very 'webmail services' dependant... I think of a kind of 'plug-in mode' witch could permit to add webmail facility 'on the fly' (who said "he's dreaming" ?)|
|User Panel configuration||The possibility for each user to set alone his/her fetchmail rule, without access to the other account configuration. This could/should be included in the Dungog userpanel|
|Some fine-tune options in the Panel||Some options (like the name of the internal mail server) are today 'hard-coded'. This is bad... Theses options should at least be stored in the configuration database (thus permitting change without need to go 'deep in the files'), and could eventually be on the server-manager panel.|
|Better 'SME Spirit' integration||The installation process of smeserver-fetchmail is not in the good SME-Spirit. This prevents smeserver-fetchmail from been included in a first installation CD.|
|Config reporter||The fetchmail configuration process is not ever so straightforward. It could be 'cool' to have a 'report configuration' panel, giving in one page wich mail account(s) is/are fetched via wich SME account, and who are the final recipients. (Actually, it could be cool for me, when questions arise !).|
|Your great idea here||Your marvellous explanations here.|
The initial sme-fetchmail was written by Vincent Filali (his site). The whole fetchmail concept is from him.
The current version, smeserver-fetchmail is a complete interface rework, but the base engine is still the same. This work and the documentation was done by me, Pascal Schirrmann (my site), under supervision of Vincent.
|April the 20, 2008||bug correction (version 1.0.11) working together with smeserver-fetchmail and smeserver-fetchmail-utf81.3.5-02 rpms.
SME did with the April 2008 update a big step, going from ISO-8859-1 charset to UTF-8. But, as not everybody update his server at the same time, Contribs have to work in the two worlds. So there are now two packages, smeserver-fechmail and smeserver-fetchmail-utf8.
|March the 20, 2007||bug correction (version 1.0.10) working together with smeserver-fetchmail-1.3.5-01 rpms.
Some translations typo stop the display of fetchmail in the server-manager panel
|March the 8, 2007||bug correction (version 1.0.9) working together with smeserver-fetchmail-1.3.5-00 rpms.
Use of new SME server modifications. Now, incoming fetched mails are again checked by the anti spam filter.
(Still experimental) support of POP3 and IMAP4 via SSL.
Control of the fetchmail timeout value, in case of a non responding remote server (default wait time : 60 seconds).
Warning: this version can only be installed on SME server starting at version 7.1.1 !
|October the 25, 2006||bug correction (version 1.0.8) working together with smeserver-fetchmail-1.3.4-10 rpms.
Small bug correction : SME 7 now accept user names with dots ( . ) in it. smeserver-fetchmail didn't like that !
|April the 29, 2006||Some small corrections and adds (version 1.0.7) working together with smeserver-fetchmail-1.3.4-09 rpms.
Brand new german version, thanks to Dietmar Berteld.
IMAP is now the default protocol for new fetchmail accounts: IMAP works better than POP3 with fetchmail, and most mail servers have IMAP capabilities.
In fact, smeserver-fetchmail didn't work with SME 7! It only worked because of file expand due to other servicies. Now, smeserver-fetchmail works totally (and bug free, of course :-) ) on SME7.
|January the 14, 2006||Some small corrections and adds (version 1.0.6) working together with smeserver-fetchmail-1.3.4-08 rpms.
External mail servers can now by given as name or IP address.
Brand new english version, thanks to Dave Dumolo.
Minor changes to hopefully suppress all errors messages during upgrade and/or uninstall on SME 7 version.
|October the 22, 2005||Some small corrections and adds (version 1.0.5) working together with smeserver-fetchmail-1.3.4-07 rpms.
smeserver-fetchmail didn't work with SME 7 beta 4 and newer, due to a change in the FormMagick library.
Warning : the SME 7 support is still very light : there's still errors messages when upgrading or uninstalling the contrib, and I still wasn't able to do all needed sanity tests.
|August the 19, 2005||Some small corrections and adds (version 1.0.4) working together with smeserver-fetchmail-1.3.4-06 rpms.
This release contains a bug fix : Preceding version incorrectly send incoming mails to the SmartHost, if a SmartHost has been set. But the smart host is set for Outgoing mails !
|August the 2, 2005||Some small corrections and adds (version 1.0.3) working together with smeserver-fetchmail-1.3.4-05 rpms.
This release contains theses changes :
|May the 20, 2005||1.0.2 : Add a 'wishList' section|
|May the 15, 2005||Some very small corrections (version 1.0.1) working together with smeserver-fetchmail-1.3.4-04 rpms.
This release contains theses three changes :
|May the 16, 2004||Initial version (1.0) corresponding to RPMs smeserver-fetchmail-1.3.4|
|copyleft Schirrms Studio 2002-2004||
||The whole website www.schirrms.net is under GFPL license|