- dovecot - dovecot.org

[Dovecot] Shared Squat index for public mailboxes
by Frerich Raabe 11 Oct '12

11 Oct '12

Hi, I'm running Dovecot 1.2.17 for serving mail via IMAP as well as for providing access to a mailing list archive. The archive is implemented as a public read-only mailbox with per-user index files (i.e. the \Seen flags are per-user). I recently enbled the Squat plugin to accelerate searches in the message bodies and noticed that every user (I'm using a virtual user setup) gets his own dovecot.index.search and dovecot.index.search.uids copies. Is it possible to share those files among all users of the system? The squat plugin appears to store the search indices among the other index files (as explained on http://wiki.dovecot.org/Plugins/FTS/Squat) no matter what; I considered storing a central copy of the index files somewhere and then creating symlinks for all users. It should be ok as far as file-permissions go since all mail is owned by a single vmail system user, but I wonder whether the indices are really the same (I noticed their md5 checksums differ) and whether there may be file locking issues in case two users search message bodies simultaneously. Can anybody shed some light? -- Frerich Raabe - raabe(a)froglogic.com www.froglogic.com - Multi-Platform GUI Testing

3 7

[Dovecot] dovecot cores
by dovecot＠freakout.de 11 Oct '12

11 Oct '12

Hi dovecot-community, i have set up a new dovecot server - everything is self-compiled and with newest versions. dovecot daemon dies every night: Bugreport Mail - dovecot cores with sig11 my dovecot daemon cores every night - no message in log file. Please help. It runs in an OpenVZ container - therefore reiserfs is reported as filesystem. root@glen ~]# cat /etc/dovecot/dovecot.conf # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.8.2.el5.028stab101.1PAE i686 CentOS release 5.8 (Final) reiserfs auth_mechanisms = plain login cram-md5 apop default_login_user = nobody disable_plaintext_auth = no first_valid_uid = 300 listen = * mail_gid = 332 mail_home = /var/dovecot mail_location = maildir:/var/spool/mail/virtual/%d/%n mail_plugins = quota mail_uid = 332 passdb { driver = sql args = /etc/dovecot/sql.conf } plugin { quota = maildir:User quota } protocols = imap pop3 ssl_cert = </etc/exim/glen.ypack.de.pem ssl_key = </etc/exim/glen.ypack.de.key userdb { driver = prefetch } userdb { driver = sql args = /etc/dovecot/sql.conf } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = yes pop3_uidl_format = %08Xu%08Xv } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota } root@glen ~]# cat /etc/dovecot/sql.conf driver = mysql connect = host=much dbname=toarx user=exim password=xxxxxxxx default_pass_scheme = PLAIN password_query = select login as username, password, dbuser_uid, dbuser_gid, dbuser_home, dbuser_mail, userdb_quota_rule from vusers where email = '%u' user_query = select login as username, password, dbuser_uid, dbuser_gid, dbuser_home, dbuser_mail, userdb_quota_rule from vusers where email = '%u' iterate_query = select email as user from vusers where user != 'odmr' root@glen run]# gdb /opt/dovecot/sbin/dovecot /var/dovecot/run/core.20049 GNU gdb (GDB) CentOS (7.0.1-42.el5.centos.1) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/dovecot-2.1.10-3/sbin/dovecot...(no debugging symbols found)...done. [New Thread 20049] Reading symbols from /opt/mysql/lib/libmysqlclient.so.18...done. Loaded symbols for /opt/mysql/lib/libmysqlclient.so.18 Reading symbols from /opt/dovecot/lib/libdovecot.so.0...done. Loaded symbols for /opt/dovecot/lib/libdovecot.so.0 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /opt/mysql/lib/libstrings.so...done. Loaded symbols for /opt/mysql/lib/libstrings.so Reading symbols from /opt/zlib/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /opt/zlib/lib/libz.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /opt/ssp/lib/libssp.so.0...done. Loaded symbols for /opt/ssp/lib/libssp.so.0 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Core was generated by `/opt/dovecot/sbin/dovecot'. Program terminated with signal 11, Segmentation fault. #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 (gdb) bt #0 0xb7f95fe4 in str_to_time () from /opt/mysql/lib/libmysqlclient.so.18 #1 0xb7f131c0 in master_instance_list_add_line (list=0x9d48880, line=0x9d540c8 "1349762052\tdovecot\t/var/dovecot/run\t") at master-instance.c:85 #2 0xb7f1331f in master_instance_list_refresh (list=0x9d48880) at master-instance.c:115 #3 0xb7f13620 in master_instance_write_init (list=0x9d48880, dotlock_r=0xbff6fb60) at master-instance.c:173 #4 0xb7f1390d in master_instance_list_set_name (list=0x9d48880, base_dir=0x8055300 "/var/dovecot/run", name=0x8055323 "dovecot") at master-instance.c:252 #5 0x0804c347 in net_accept () at network.c:502 #6 0xb7f3255b in io_loop_handle_timeouts_real (ioloop=0x9d465b0) at ioloop.c:354 #7 0xb7f325de in io_loop_handle_timeouts (ioloop=0x9d465b0) at ioloop.c:367 #8 0xb7f338b6 in io_loop_handler_run (ioloop=0x9d465b0) at ioloop-poll.c:171 #9 0xb7f326d3 in io_loop_run (ioloop=0x9d465b0) at ioloop.c:398 #10 0xb7f17304 in master_service_run (service=0x9d464e0, callback=0) at master-service.c:543 #11 0x0804d375 in net_accept () at network.c:502 #12 0xb7d84e9c in __libc_start_main () from /lib/libc.so.6 #13 0x0804b501 in net_accept () at network.c:502 (gdb) q Regards Axel

2 1

[Dovecot] memory allocation in new thread
by . . 11 Oct '12

11 Oct '12

Hi! I have some problems with memory allocation. I create new thread in cidir storage and call malloc(), and it fails to allocate even 1 byte. What can cause this problem? Dovecot vesrion is: 2.1.10 (130563b592c9+) Sample code looks like this (I also link to pthread with: -pthread): #define TEST_MALLOC() \ { \ void *p; \ p = malloc(1); \ if (!p) { \ i_info("%s: malloc() failed", __FUNCTION__); \ } else { \ i_info("%s: malloc() succeeded", __FUNCTION__); \ free(p); \ } \ } #include <pthread.h> static void *test_pthread_malloc_func(void *data_) { TEST_MALLOC(); return NULL; } static void test_pthread_malloc() { int ret; pthread_t tid; TEST_MALLOC(); ret = pthread_create(&tid, NULL, test_pthread_malloc_func, NULL); if (ret) { i_info("failed to start thread"); } else { pthread_join(tid, NULL); } } I call test_pthread_malloc() function. It produces following output: Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc: malloc() succeeded Oct 11 12:56:15 imap(guest): Info: test_pthread_malloc_func: malloc() failed

2 1

[Dovecot] Irrelevant information filling logs
by 3.listas＠adminlinux.com.br 11 Oct '12

11 Oct '12

Hi, I have a "Ubuntu10.04 + dovecot-2.0.13" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. There is a lot of logs of "quota exceeded" like this: Oct 10 13:00:56 mailboxserver5 dovecot: lmtp(29105, user@mailboxserver5): Error: ifcIN1NxdVCxcQAAMBx7mQ: sieve: msgid=unspecified: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) These messages are not important to me. But these messages fill the log files, damaging the display of messages that could be important. Is there a way to send specific Dovecot errors on specific files or just discard them? Thanks -- Thiago Henrique adminlinux.com.br

6 5

[Dovecot] some questions on AOX or rather a mail system setup
by Christoph Anton Mitterer 11 Oct '12

11 Oct '12

Hi folks. Perhaps you find some time to look into this,... I'm trying to plan my mail system and would have some questions. The overall idea is about the following: - There is a (internet) server, which is the MTA (which will be postfix) and imap and/or pop3 server (which shall be, guess, dovecot ... AOX comes in later). - Any spam filtering, virus-scanning, mail filtering happens on that server. - I have not yet decided on whether to use maildrop for this or Sieve. Maildrop is nice, but has one major deficiency which I don't know how to work around. - A few clients (that means I won't serve 100 of users) connect to that server via imap and should see all mails, etc. already in some fancy sorted hierarchy (that means filtered into different directories). So far nothing complicated. But now... - Apart from spam, I never delete mail; and because I'm subscribed to many lists, I get a lot of mail. - Storage on my server is limited and it's located somewhere at my ISP, so I generally do not trust it with respect to safety... For both reasons, I want the canonical archive of all mail to be at home at some local server. - The local server should also be an imap server, so that I can access the archive from may computers at home. - The local server won't be available from the internet. - The local archive should have the same folder hierarchy as the internet server (I'd prefer not to filter twice). - The local archive might be AOX... Mail Flow: - I generally want to have _all_ mail (which is not sorted out because of being spam) to be archived at the local server. - But(!) I want to selectively keep (in addition) mail at the internet server. For example I may want to select the folder that contains all mail form some friend to be kept online completely. But I may want to decide that mailinglists keep only the last 10 days and/or 1000 messages of mail. - The idea is, that the local server regularly (when it is online/running) catches new mail from the internet server... and stores it in the archive. - So apart from new mail that has not yet been read, that local archive contains always all mails that are also on the internet server... the later may contain (for specific directories) the same, or just parts of. - The MUAs will then have two imap accounts, one to the internet server and one to the local archive,... each one being usable, depending on where I am. 1) This is where my first problem arises: How can I implement that mail flow, especially: - How do I secure that all mail is read from the internet server (i.e. that nothing is "forgotten"? - How do I make sure that no mails are retrieved twice (or more)? A problem which I often had with pop, when the mail client crashed during sync? - Further it must be secured, that when I delete something on the internet server, it is NOT deleted on the local server (on the next mail-fetching).... this is why I don't use the word "sync". a) One stupid solution would be, that I duplicate all mail on the online server,... one part is for staying online, one part is for being fetched to the local archive. As soon as it was fetched... that copy gets removed (always). That solution would give a clean and secured separation of both? b) I don't think offlineimap or any other caching-like solution is the right thing... especially as one must always fear that such a cache may be accidentally wiped. Are there better solutions than (a)? 2) Problem would be already a refinement of a working solution for (1) (but obviously not when using (1).(a) ). When e.g. reply to or forward a mail using the online server,... and that mail had already been fetched,... can I make the flag synced? 3) Is AOX suitable for the local server? - I couldn't use maildir locally, because I loose just to much space to the block fragmentation. But I guess I won't have this problem with the DB backend. a) Is AOX able to manage a really big folder hierarchy that basically ever keeps growing... with easily several 100k mails per folder... and that is in total already over 100GB? I read that e.g. dovecot would have more performance problems with that. b) I would prefer to have fast full text search. Does AOX provide this? I read that IMAP has limitations which make full text search not really usable via it? Further, I remember some dovecot wiki page that showed a comparison which said that both do not perfectly implement imap. Thanks a lot, Chris.

3 5

[Dovecot] iterate_query does not use userdb - mail_location not found
by dovecot＠freakout.de 11 Oct '12

11 Oct '12

Hi dovecot-comminity, can't get iterate_query working. doveadm cannot find mail_location which comes from userdb query. dovecot itself works fine with sql. Whats wrong? please help: [root@glen exim]# /opt/dovecot/bin/doveadm search -A mailbox Trash savedbefore 90d doveadm(uwe(a)mitmachnet.de): Error: user uwe(a)mitmachnet.de: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=(not set) doveadm(uwe(a)mitmachnet.de): Error: User init failed dovecot.conf: passdb { driver = sql args = /etc/dovecot/sql.conf } userdb { driver = sql args = /etc/dovecot/sql.conf } sql.conf: driver = mysql connect = host=much dbname=toarx user=exim password=xxxxxxxx default_pass_scheme = PLAIN password_query = select user as username, password, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' user_query = select user as username, userdb_home, userdb_mail, userdb_quota_rule from vusers where email = '%u' iterate_query = select email as user from vusers Cheers Axel

2 1

[Dovecot] Fwd: Re: Dovecot doesnot disconnect at end of pop3 session
by Robert JR 10 Oct '12

10 Oct '12

On 2012-10-10 17:47, Jack Bates wrote: > On 10/10/2012 9:40 AM, Robert JR wrote: > >> telnet mymail.com 110 user username pass password list quit all >> commands work except the quit command, Dovecot hang at this point >> !!! >> Why Dovecot hang on receiving the Quit command !!! and doesnot >> disconnect !! > Please include your dovecot version. Also, does the list command end > with a line that contains a period? You might want to include dovecot > -n > as well, which will show everyone what your mail storage type is and > other relevant information. Jack # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.9.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login auth_username_format = %Lu disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mal:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } protocol pop3 { mail_max_userip_connections = 4 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } Please find the output of dovecot -n list command return msg number and size only. Thanks

2 1

[Dovecot] Dovecot doesnot disconnect at end of pop3 session
by Robert JR 10 Oct '12

10 Oct '12

Hello, I have 3 users checking same email account using pop3 (outlook express) suddenly after some hours , users are unable to disconnect from pop3 after checking mail So i telnet to the pop3 and found that the issue is true but i do not know the reason telnet mymail.com 110 user username pass password list quit all commands work except the quit command, Dovecot hang at this point !!! Why Dovecot hang on receiving the Quit command !!! and doesnot disconnect !! Any help is greatly appreciated. Thanks.

2 1

Re: [Dovecot] I need a quota expert
by Tibby 10 Oct '12

10 Oct '12

I'm having problems with dovecot quota configuration. If I Include the driver mysql in dovecot-sql.conf I'm getting this error: dovecot: dict: Error in configuration file /etc/dovecot/dovecot-sql.conf line 1: Unknown setting: driver dovecot: dict: Failed to initialize dictionary 'quotadict' If I Don't include it: dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf dovecot: auth(default): Fatal: sql: driver not set in configuration file /etc/dovecot/dovecot-sql.conf Now What??? How Should I configure quota ? My SQL has a mail db with a user table. The table has email, password, quota_kb fields. my dovecot.conf: ---------------- protocols = imap imaps pop3 pop3s log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir disable_plaintext_auth = no ssl_cert_file = /etc/ssl/certs/dovecot.pem ssl_key_file = /etc/ssl/private/dovecot.pem namespace private { separator = . prefix = INBOX. inbox = yes } protocol imap { mail_plugins = quota imap_quota } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster(a)domain.tld mail_plugins = sieve quota global_script_path = /home/vmail/globalsieverc mail_plugin_dir = /usr/lib/dovecot/modules/lda } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } dict { quotadict = mysql:/etc/dovecot/dovecot-sql.conf } plugin { quota = dict:user::proxy::quotadict quota_rule = *:storage=10M:messages=1000 } auth default { user = root passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0600 user = vmail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } mail_uid = 5000 mail_gid = 5000 my dovecot-sql.conf: ---------------- driver = mysql connect = host=127.0.0.1 dbname=mail user=****** password=****** default_pass_scheme = CRYPT password_query = SELECT email as user, password, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u'; user_query = SELECT CONCAT('/home/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota_kb) AS quota_rule FROM users WHERE email='%u' map { pattern = priv/quota/storage table = user username_field = user value_field = quota_kb } SOMEBODY PLEASE HELP ME!!! :)

2 1

[Dovecot] About ManageSieve and TLS
by 3.listas＠adminlinux.com.br 10 Oct '12

10 Oct '12

Hi, I have a "ubuntu10.04 + dovecot-2.0.13" configuration in my server. It works fine with ~50k accounts. Recently I enabled TLS: $ cat /etc/dovecot/dovecot.conf ... # Use SSL ? ssl = yes ... The goal was to provide only IMAPS and POP3S. But Managesieve says "STARTTLS": $ telnet _MY_IP_ 2000 Trying _MY_IP_... Connected to _MY_IP_. Escape character is '^]'. "IMPLEMENTATION" "K8 ManageSieve" "SIEVE" "comparator-i;ascii-numeric copy envelope fileinto imapflags include notify regex reject relational subaddress vacation" "SASL" "PLAIN LOGIN" "STARTTLS" "VERSION" "1.0" OK "K8 IMAP/POP3 server" doveconf -a shows: service managesieve-login { ... inet_listener sieve { address = _MY_IP_ port = 4190 ssl = no } inet_listener sieve_deprecated { address = _MY_IP_ port = 2000 ssl = no } ... } I think there is something wrong there but I don't know. I think Managesieve should not say "STARTTLS". Can someone help me? Thanks. -- Thiago Henrique adminlinux.com.br

2 2