Home

So, you would like to upgrade your Debian Sarge box from amavisd-new version 20030616-p10 to amavisd-new 2.4.4. You might ask why I don't simply install the new Debian version of 2.4.2; here is why. Version 2.4.4 requires libcompress-zlib-perl version 1.35 or newer but the latest version available in 'stable' is 1.34. If you were to upgrade libcompress-zlib-perl to the 'testing' version, it may also upgrade libc6, libc6-dev, locales, libdb4.3, perl, perl-base and perl-modules to 'testing'. This may not be desirable and can be avoided by installing Compress::Zlib from source or from backports.org. In fact, installing programs from 'testing' or 'unstable' could potentially remove your kernel. If you are running a 2.6.8 or earlier kernel at the present time, and you wish to install programs from 'testing' or 'unstable' that depend on a newer version of libc6 at any time in the future, it is quite possible you are going to have to migrate to one of the new linix-image kernels. See http://www200.pair.com/mecham/spam/kernel.html

You can install libcompress-zlib-perl (Compress::Zlib) from source code (or from backports.org) which will enable us to use newer versions of amavisd-new without installing packages from 'testing'. Here's how:
Note that an alternative to the following would be to install libcompress-zlib-perl from http://www.backports.org/dokuwiki/doku.php?id=instructions. In fact, I prefer to install libcompress-zlib-perl from backports.org. If you do, in order to keep 'apt-get update' from crashing, you will probably need to create an /etc/apt/apt.conf file with the contents  APT::Cache-Limit "25165824";

cd /usr/local/src
wget http://search.cpan.org/CPAN/authors/id/P/PM/PMQS/Compress-Zlib-1.41.tar.gz
tar xzvf Compress-Zlib-1.41.tar.gz
cd Compress-Zlib-1.41
perl Makefile.PL
make && make test && make install
cd


If all goes well, the last thing you will see is: "Appending installation info to /usr/local/lib/perl/5.8.4/perllocal.pod"
Note that when we manually install Perl programs like this, the Debian version will be ignored because Perl will find this program first.

Amavisd-new 2.4.4 has many more configuration options available making it a more flexible yet more complex system. I have to admit, there are some configuration options I do not understand. The main reason I wanted to upgrade is because it is now possible to send email containing banned files or bad headers to quarantine areas that are no longer common to the virus quarantine area. This new version of amavisd-new also has better integration with SpamAssassin, better support on the mailing list, adds new features and is more secure.

There are innumerable ways to approach this upgrade. The method I have chosen will use a 'customized for Debian' version of amavisd.conf as a basis for the new configuration. While this new version of amavisd-new will likely run on a completely unmodified 20030616-p10 configuration file, doing so will likely mask problems (like quarantining banned files to a local file instead of sending them to a mailbox). There are a few new settings that will have to be added to our new configuration file to make amavisd-new work properly and some old configuration settings will get replaced with new 2.4.4 style settings because they work better. Upgrading this way will introduce us to many of the new features and familiarize us with the syntax of 2.4.4. If you use Maia Mailguard, this upgrade will of course kill it. This upgrade assumes you are not using LDAP or SQL for data tables but it should still work fine if you are (as long as you carry those settings over to the new amavisd.conf). It also assumes you built your spamfilter using one of my documents at http://www200.pair.com/mecham/spam/archives.html.

Here is a general overview of the process:
  • 1. Start with a fully configured and working 20030616-p10 installation.
  • 2. Download the source code for version 2.4.4.
  • 3. Make backup copies of /etc/amavis/amavisd.conf and /usr/sbin/amavisd-new.
  • 4. Create a scratch copy of our current amavisd.conf with all the comments removed to make it easier to read.
  • 5. Insure all of the external modules amavisd-new uses are up to date.
  • 6. Modify as needed a custom amavisd.conf you download from me (amavisd.conf-2.4.4) by comparing it side-by-side to our scratch file.
  • 7. Stop amavisd-new
  • 8. Replace /etc/amavis/amavisd.conf and /usr/sbin/amavisd-new with the new files.
  • 9. Start amavisd.new.
  • 10. If it crashes, stop amavisd-new, restore from backup, start amavisd-new.

    Make sure we are prepared for some of the new features:

    mkdir /var/lib/amavis/tmp
    mkdir /var/lib/amavis/db
    mkdir /var/lib/amavis/var
    ln -s /etc/amavis/amavisd.conf /etc/amavisd.conf
    chown -R amavis:amavis /var/lib/amavis
    chmod -R 750 /var/lib/amavis


    In this section, we download source, make multiple backup copies of our 20030616-p10 files and download a custom version of amavisd.conf (amavisd.conf-2.4.4) which is the configuration file we will work with until we are ready to actually do the upgrade:

    cd /usr/local/src
    wget http://www.ijs.si/software/amavisd/amavisd-new-2.4.4.tar.gz
    tar xzvf amavisd-new-2.4.4.tar.gz
    rm amavisd-new-2.4.4.tar.gz
    cd amavisd-new-2.4.4
    cp amavisd amavisd-new
    cp amavisd-new /usr/sbin/amavisd-new-2.4.4
    cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-20030616
    cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-20030616-extra-backup
    cp /usr/sbin/amavisd-new /usr/sbin/amavisd-new-20030616
    cp /usr/sbin/amavisd-new /usr/sbin/amavisd-new-20030616-extra-backup
    cd /etc/amavis
    wget http://www200.pair.com/mecham/amavisd/2.4.4/amavisd.conf-2.4.4
    wget http://www200.pair.com/mecham/amavisd/2.4.4/amavisd.conf-sample
    grep -vE "^$|^#" /etc/amavis/amavisd.conf > /etc/amavis/amavisd.conf-20030616-short


    It's a good idea to make sure all of the external modules amavisd-new requires are up to date. You should exercise caution here because in order to solve dependencies, apt-get may upgrade Perl. This is at times a bad thing, but I personally have had no problem when it does. You may possibly need to reinstall any Perl modules you installed via CPAN if apt-get upgrades Perl. I suggest you have a good backup of your hard drive before you continue. You can first simulate the installation by using 'apt-get -s install [programs]'.

    apt-get update

    apt-get install file libarchive-tar-perl libarchive-zip-perl libconvert-tnef-perl libconvert-uulib-perl libberkeleydb-perl libmailtools-perl libdb4.3-dev libmime-perl libnet-perl libnet-dns-perl libconvert-binhex-perl libnet-server-perl libunix-syslog-perl arj nomarch unrar pax cabextract libmail-spf-query-perl libdigest-nilsimsa-perl libnet-ident-perl libnet-ph-perl libnet-snpp-perl


    I suggest you read the amavisd-new RELEASE_NOTES, most notably the release notes for amavisd-new-20040701 which was the first of the 2.0 versions. This section will illustrate the most important changes since 20030616-p10 and will explain some of the new features of the 2.0 variants.

    If you are a Windows user like me, this upgrade will go much easier by using WordPad (or Notepad) to display the old amavisd.conf file, and the WinSCP editor to edit /etc/amavis/amavisd.conf-2.4.4. Open up WinSCP and edit /etc/amavis/amavisd.conf-20030616-short. Select the all the text and copy it to the clipboard, then close the editor. Open up WordPad and paste the entire contents into it. You may optionally save the document. Size the WordPad window so it fills the left hand side of your screen. Size the WinSCP window so it fills the right hand side of your screen. Now open up /etc/amavis/amavisd.conf-2.4.4 and size it to also fit the right hand side of your screen. So, obviously the idea is to transfer settings from the old file on the left hand to the new file on the right hand. Below is step-by-step notation of the parameters that were changed in my amavisd.conf-2.4.4 file during this process, yours may/will be different:

    $MYHOME
    $mydomain
    $myhostname
    If you have not set $myhostname before, you should set it now. Example would be 'sfa.example.com'.

    $daemon_user
    $daemon_group
    $pid_file
    $lock_file

    $enable_db (new setting)
    $enable_global_cache (new setting)
    Note: If you get errors when you first start your 2.4.4 program up, the first thing I would do is disable those two new settings. However, if you leave these settings disabled you will loose some new features of 2.x.x (like amavisd-nanny). It is also a known problem that on a busy server, BerkeleyDB may run out of locker entries and as a result amavisd-new will begin refusing mail, causing mail to back up in the Postfix deferred queue. This problem appears to be solved by using libconvert-uulib-perl version 1.05 or newer.

    $max_servers
    $max_requests

    Instead of the old @local_domains_acl, I used the new @local_domains_maps in the form:
    @local_domains_maps = ( [ '.example.com', '.example2.com', '.example3.com' ] );

    $relayhost_is_client (no need to bring this over)
    $insert_received_line
    $unix_socketname
    $inet_socket_port
    $inet_socket_bind
    @inet_acl (use the new default for @inet_acl not the old one)

    $LOGFILE
    $log_level
    is your choice. Set it to 2 to get useful log files unless you are a busy site and need to reduce the size of the log files or simply don't need the extra information. In that case, set to 0. I set mine to 0 to avoid frequent logcheck reports.

    Do not use the $log_templ section from your old config file.

    read_l10n_templates
    The Debian package maintainers created notification templates in different languages in subdirectories in /etc/amavis. If you would like to continue to use these templates, copy over the read_l10n_templates parameter from your old config. If you choose not to use them, amavisd-new will use the default (English) templates at the bottom the amavisd-new source code. The external templates that are provided by Debian are based on version 20030616p10 so they are somewhat dated. If you would like to continue to use external templates, but would like to update them to the versions supplied with amavisd-new version 2.4.4, you can optionally download the files from me (English only at this time). I provide instructions at the bottom of this page.

    $final_virus_destiny
    $final_banned_destiny
    $final_spam_destiny
    $final_bad_header_destiny

    Rather that use the old $viruses_that_fake_sender_re, use the new @viruses_that_fake_sender_maps provided.

    $virus_admin (and possibly $spam_admin) Add a new line below this, something like:
    $banned_admin = "postmaster@\$mydomain";

    $mailfrom_notify_admin
    $mailfrom_notify_recip
    $mailfrom_notify_spamadmin
    $hdrfrom_notify_sender
    $QUARANTINEDIR
    $virus_quarantine_to

    $banned_quarantine_to (new setting)
    $bad_header_quarantine_to (new setting)
    These new parameters must be configured. I suggest creating a new mailbox to send messages containing banned attachments to, "banned\@$mydomain", and treat them in a similar manner to spam messages. You can use the same mailbox for bad headers. Set your 'banned' email client's SMTP server to your final destination server (your Exchange server) so when you forward mail out of this mailbox, it will not pass through the spamfilter again.

    $spam_quarantine_to
    $X_HEADER_TAG
    $X_HEADER_LINE

    $defang_virus (new setting)
    $defang_banned (new setting)
    These will wrap up the message into an attachment and give the recipient a warning about opening the attachment. It does not make the attachment safe to open. I personally do not defang, because I quarantine these messages and personally review them. Messages that end up in a quarantine do not get defanged.

    Use the new @keep_decoded_original_maps instead of the old $keep_decoded_original_re

    $banned_filename_re
    has changed quite a bit. Add or remove items to meet your needs. Now is a good time to review what it is you ban.

    If you have virus_lovers or spam_lovers, you may wish to use the new @virus_lovers_maps and @spam_lovers_maps - read the examples in amavisd.conf-sample for details. Also read about the new syntax of bypass_virus and bypass_spam.
    You may need to set   @bypass_virus_checks_maps = (1);   if you wish to disable virus scanning.

    $recipient_delimiter
    $replace_existing_extension

    $blacklist_sender_re and $whitelist_sender have been replaced with @score_sender_maps.
    This new method uses "soft" white or black listing by modifying the SpamAssassin score by the numbers shown in the example. I suggest increasing the -3.0 negative scores (to something like -6.0) to insure mail from these senders is received. If you have customized this section, move your senders from the old $whitelist_sender section and add then here. Place them below 'nobody@cert.org' => -3.0, and assign them scores as you see fit. Negative scores will "soft" whitelist them, and positive scores can be used to "soft" blacklist senders.


    Do NOT carry this group of settings over to 2.4.4. Some of this has been replaced by a new @decoders list.
    Some of the settings have changed so we will simply use the new settings as supplied with 2.4.4.
    $MAXLEVELS
    $MAXFILES
    $MIN_EXPANSION_QUOTA
    $MAX_EXPANSION_QUOTA
    $MIN_EXPANSION_FACTOR
    $MAX_EXPANSION_FACTOR
    $path
    $file
    $gzip
    $bzip2
    $lzop
    $uncompress
    $unfreeze
    $arc
    $unarj
    $unrar
    $zoo
    $lha
    $cpio
    $sa_timeout


    Now, carry over some of our most important settings:
    $sa_local_tests_only
    $sa_tag_level_deflt
    $sa_tag2_level_deflt
    $sa_kill_level_deflt
    $sa_dsn_cutoff_level

    $sa_quarantine_cutoff_level
    This is a new setting that allows you delete high scoring spam that is destined to be quarantined. If you previously used my amavisd-new hack for deleting high scoring spam, this setting can be used instead.

    In the @av_scanners section, configure AV scanners as you did in the old file. If you use ClamAV, make sure you edit the value after CONTSCAN so it matches your old config and remember to uncomment the ClamAV section. I personally delete all the virus scanners I don't use. I keep ClamAV and BitDefender and delete or comment out all the ones in between these two. The only reason I do this is because it lessens the amount of data displayed when running amavisd-new debug. Remember that you must leave at least one backup scanner uncommented, whether it is used or not.

    OK, save your file, then let's give it a try:

    amavisd-new stop
    cp /usr/sbin/amavisd-new-2.4.4 /usr/sbin/amavisd-new
    cp /etc/amavis/amavisd.conf-2.4.4 /etc/amavis/amavisd.conf
    amavisd-new debug


    Watch for errors - send test mail through as usual.
    If everything looks great, Ctrl+c then:

    amavisd-new start

    Enjoy.

    If things don't go well, you will want to revert to 20030616-p10 until your problems are solved:

    amavisd-new stop
    cp /usr/sbin/amavisd-new-20030616 /usr/sbin/amavisd-new
    cp /etc/amavis/amavisd.conf-20030616 /etc/amavis/amavisd.conf
    amavisd-new start


    If you are running logcheck, you most likely will not want to be annoyed by every message amavisd-new blocks,
    so edit this file using vi or the WinSCP editor:

    vi /etc/logcheck/ignore.d.server/amavisd-new

    At the bottom, you will find the text:
    amavis\[[0-9]+\]: +(\([-0-9]+\) +)?(SPAM|Not-Delivered|Passed|BANNED|INFECTED)

    Change this to:
    amavis\[[0-9]+\]: +(\([-0-9]+\) +)?(Blocked|SPAM|Not-Delivered|Passed|BANNED|INFECTED)

    If all goes well, make a backup copy of our new configuration file:

    cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-2.4.4-backup


    Since running   apt-get upgrade   or   apt-get install amavisd-new   will clobber our new installation, either don't do either of these, or revert back to 20030616-p10 during the upgrade, or use the instructions below to prevent amavisd-new from upgrading. Either way, just make sure you keep backup copies of each version of amavisd.conf and amavisd-new.

    To prevent the old version of amavisd-new from installing over our new one, place the package on hold:
    echo "amavisd-new hold" | dpkg --set-selections

    In the future, if you should ever need to take amavisd-new off hold:
    echo "amavisd-new install" | dpkg --set-selections

    Optional:
    I suggest actually removing the Debian version of amavisd-new in order to prevent an accidental upgrade.
    Begin by backing up a few files:


    mkdir /root/ambackup
    cp /usr/sbin/amavisd-new /root/ambackup
    cp /etc/amavis/amavisd.conf /root/ambackup
    cp /etc/init.d/amavis /root/ambackup


    Run this to make sure amavisd-new is the only thing that will be removed:
    apt-get -s remove amavisd-new

    If it is not the only thing that will be removed, then make a note of any other programs that will get removed (because you will have to reinstall them) and proceed with extreme caution - at your own risk. Hopefully, only amavisd-new will be removed. If that is true, then you can remove it:
    apt-get remove amavisd-new

    Now amavisd-new will be broken (the guts are missing), but it's easy to fix:
    cp /root/ambackup/amavisd-new /usr/sbin/amavisd-new
    ln -s /etc/amavis/amavisd.conf /etc/amavisd.conf
    cp /root/ambackup/amavis /etc/init.d
    chmod +x /root/ambackup/amavis

    Once you have checked that you still have an /etc/amavis/amavisd.conf then you can start up amavisd-new:
    /etc/init.d/amavis start


    Optional:
    If you would like to enable the external template files so you may customize them in the future (English only):
    vi /etc/amavis/amavisd.conf

    and create or uncomment the line (careful, there may a another similar line):
    # read_l10n_templates('en_US', '/etc/amavis');

    Then replace the outdated files with more recent ones, begin by making a backup copy of the old ones:
    cp -r /etc/amavis/en_US /etc/amavis/backup-en_US

    Then remove the old files and install new files that I have created:

    cd /etc/amavis/en_US
    rm charset
    rm template-dsn.txt
    rm template-spam-admin.txt
    rm template-spam-sender.txt
    rm template-virus-admin.txt
    rm template-virus-recipient.txt
    rm template-virus-sender.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/charset
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-dsn.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-spam-admin.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-spam-sender.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-virus-admin.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-virus-recipient.txt
    wget http://www200.pair.com/mecham/amavisd/2.4.4/en_US/template-virus-sender.txt


    Last updated 15 OCT 2006, mr88talent at yahoo dot com