[Dovecot] Problems with dovecot 2.1.7, spamassassin 3.3.2 and antispam plugin

[Dean]

Darn.  Thought I had found a good point about pthread_cancel.  See the
last post here :

    https://bugs.launchpad.net/ubuntu/+source/gcc-3.3/+bug/40285

    Juergen Perlinger (juergen-perlinger)
    <https://launchpad.net/%7Ejuergen-perlinger> wrote on 2013-10-25:
    #30
    <https://bugs.launchpad.net/ubuntu/+source/gcc-3.3/+bug/40285/comments/30>


    I finally tracked it down, I think. The pthread code tries to load
    libgcc_s on pthread_exit(), and this dos not work when the effective
    user was changed -- many daemons switch from root to a restricted
    user, and that's one way to end up in this problem. If the library
    is loaded globally for the process before the user is changed,
    everything works.

    This is a problem of the 'pthread_exit()' implementation
    (delayed/on-demand load of libgcc_s) and/or the 'mmap()'
    implementation in the kernel.


I've tried putting /lib/x86_64-linux-gnu/libgcc_s.so.1 into
/etc/ld.so.preload, I've changed the exec line in /etc/init/dovecot to

     exec LD_PRELOAD=/lib/x86_64-linux-gnu/libgcc_s.so.1
/usr/sbin/dovecot -F -c /etc/dovecot/dovecot.conf

and the sa-learn line in /usr/local/bin/sa-learn-pipe.sh to

    LD_PRELOAD=/lib/x86_64-linux-gnu/libgcc_s.so.1 /usr/bin/sa-learn -D
--progress $* /tmp/sendmail-msg-$$.txt >> /tmp/sa-learn-pipe.$$.log 2>&1

So far to no avail.  I'm going to try the spool2dir backend with
incron.  Cumbersome, but it should work ...


On 11/20/2013 02:17 PM, Dean wrote:
> On 11/18/2013 10:08 AM, Steffen Kaiser wrote:
>   
>> On Mon, 11 Nov 2013, Dean wrote:
>>     
>>>> If you are on a 64bit system, maybe sa-learn is compiled 32bit only,
>>>> then the library must be installed as 32bit version as well.
>>>>         
>>> /usr/bin/sa-learn is a perl script, calling the various
>>> Mail::SpamAssassin modules.  No 32/64 bit there afaik.
>>>       
>> I have a 64bit system and use the spamassassin demon to train
>> ham/spam without such problem. I have the 32bit lib installed:
>>
>> locate libgcc_s
>> /lib/libgcc_s.so.1
>> /usr/lib/gcc/x86_64-linux-gnu/4.3/libgcc_s.so
>> /usr/lib/gcc/x86_64-linux-gnu/4.3/libgcc_s_32.so
>> /usr/lib/gcc/x86_64-linux-gnu/4.3/32/libgcc_s.so
>> /usr/lib/gcc/x86_64-linux-gnu/4.4/libgcc_s.so
>> /usr/lib/gcc/x86_64-linux-gnu/4.4/libgcc_s_32.so
>> /usr/lib/gcc/x86_64-linux-gnu/4.4/32/libgcc_s.so
>> /usr/lib32/libgcc_s.so.1
>>
>> The 32bit library got onto the system via the "Suggests" of Debian's
>> gcc package.
>>     
>
> I installed the 32bit version too.
>
> $ locate libgcc_s.so
> /lib/x86_64-linux-gnu/libgcc_s.so.1
> /usr/lib/gcc/x86_64-linux-gnu/4.7/libgcc_s.so
> /usr/lib32/libgcc_s.so.1
>
> Still no luck.  This is with FuzzyOcr enabled, fails on the mysql DB connect
>
> 12570-sa-learn Nov 20 13:39:27.817 [12572] info: FuzzyOcr: Using scan
> gocr-180: /usr/bin/gocr -l 180 -d 2 -i $input
> 12570-sa-learn Nov 20 13:39:27.817 [12572] info: FuzzyOcr: Using scan
> tesseract: /usr/bin/tesseract $input $output
> 12570-sa-learn Nov 20 13:39:27.817 [12572] dbg: FuzzyOcr: Connecting to:
> dbi:mysql:database=FuzzyOcr;mysql_socket=/var/run/mysqld/mysqld.sock
> 12570-sa-learn libgcc_s.so.1 must be installed for pthread_cancel to work
> 12570-end
>
> And this is with FuzzyOcr disabled, so it goes right to the Bayes DB and
> fails
>
> 14826-sa-learn Nov 20 13:57:38.246 [14828] dbg: bayes: learner_new
> self=Mail::SpamAssassin::Plugin::Bayes=HASH(0x476eae0),
> bayes_store_module=Mail::SpamAssassin::BayesStore::MySQL
> 14826-sa-learn Nov 20 13:57:38.265 [14828] dbg: bayes: using username:
> debian-spamd
> 14826-sa-learn Nov 20 13:57:38.265 [14828] dbg: bayes: learner_new: got
> store=Mail::SpamAssassin::BayesStore::MySQL=HASH(0x5039840)
> 14826-sa-learn Nov 20 13:57:38.266 [14828] dbg: plugin:
> Mail::SpamAssassin::Plugin::Bayes=HASH(0x476eae0) implements
> 'learner_is_scan_available', priority 0
> 14826-sa-learn libgcc_s.so.1 must be installed for pthread_cancel to work
> 14826-end
>
> This is what is *supposed* to happen (using cmdline
> /usr/local/bin/sa-learn-pipe.sh --spam < /tmp/email-to-learn)
>
> 15412-sa-learn Nov 20 14:02:55.095 [15414] dbg: bayes: using username:
> debian-spamd
> 15412-sa-learn Nov 20 14:02:55.095 [15414] dbg: bayes: learner_new: got
> store=Mail::SpamAssassin::BayesStore::MySQL=HASH(0x3ef55f0)
> 15412-sa-learn Nov 20 14:02:55.095 [15414] dbg: plugin:
> Mail::SpamAssassin::Plugin::Bayes=HASH(0x3631ba0) implements
> 'learner_is_scan_available', priority 0
> 15412-sa-learn Nov 20 14:02:55.118 [15414] dbg: bayes: database
> connection established
> 15412-sa-learn Nov 20 14:02:55.118 [15414] dbg: bayes: found bayes db
> version 3
>
> The markasjunk2 plugin for roundcube calls sa-learn-pipe.sh and it works
> fine.  The difference is that it is called with a file parameter that
> contains the email to be learned, while Dovecot/antispam pipes the email
> into the script.  Both methods work fine, as checked on the cmdline.
>
> /usr/local/bin/sa-learn-pipe.sh --spam /tmp/email-to-learn
> /usr/local/bin/sa-learn-pipe.sh --spam < /tmp/email-to-learn
>
> It's only when it's called from the context of Dovecot/antispam that we
> see the libgcc_s.so.1 error pop up, and the mysql connection apparently
> fail.  I think that context is the key, but I don't what it is. 
> Something about the environment is causing those mysql connections to
> fail.  I believe it's permissions - the socket is 777, and the cmdline
> runs are done with a regular ID and work fine.
>
> The antispam functionality is a great feature for remote email clients
> like thunderbird.  Just need to get it working :)  Any ideas ?
>
> If anyone wants to test I can provide a small 32meg bootable ISO (Ubuntu
> mini.iso) that installs 13.04 and the various apps, all preconfigured -
> works fine for a bare machine or a VM/VPS.  I can also provide the
> installer script that does all the installs/configuration.  Run it on a
> bare 13.04 server/mini install and it does the rest.
>
>   


-- 
Dean Carpenter
deano is at areyes dot com
94TT :)