- dovecot - dovecot.org

dsync with namespaces
by Tom Sommer 23 Mar '17

23 Mar '17

I have a major dsync headache which I am hoping someone can help me with. I have to migrate mails from a 'namespace/inbox/prefix=INBOX.' server to a 'namespace/inbox/prefix=' server. So I'm trying # dsync -o 'namespace/inbox/prefix=' sync -1 -f -u xx(a)xx.com tcp:1.2.3.4 It works fine for some accounts, however for others many hundreds and sometimes thousands of mails go missing from the destination inbox, and a resync doesn't pick them up. The debug output doesn't recognize these either. If I remove the 'namespace/inbox/prefix=' part, then I get 'Error: Couldn't find namespace for mailbox X' errors. Both source and destination is running 2.2.27 I can't imapsync :S Thanks. -- Tom

1 0

Re: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings)
by Ralf Hildebrandt 23 Mar '17

23 Mar '17

* Aki Tuomi <aki.tuomi(a)dovecot.fi>: > Could you send us the gdb bt full backtrace for the core file? Currently I can't get it to create coredumps doveconf -n: # 2.2.devel (3f97702): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (023f391) # OS: Linux 4.4.0-65-generic x86_64 Ubuntu 16.04.2 LTS auth_mechanisms = plain login default_vsz_limit = 1 G imapc_host = exchange-imap.charite.de imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no listen = *,:: mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_plugins = mail_log notify mail_uid = imapproxy passdb { args = host=exchange-imap.charite.de port=993 ssl=imaps default_fields = userdb_imapc_user=%u userdb_imapc_password=%w userdb_imapc_host=exchange-imap.charite.de userdb_imapc_ssl=imaps userdb_imapc_port=993 driver = imap } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap service auth { inet_listener { address = 127.0.0.1 port = 12345 } } ssl = required ssl_ca = </etc/ssl/certs/ca-certificates.crt ssl_cert = </etc/dovecot/dovecot.pem ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } verbose_proctitle = yes Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 4

Ubuntu Dovecot 2.2.9 several Errors, Transaction log as well as BUG message
by Kein Name 23 Mar '17

23 Mar '17

Hello List, I am running an Ubuntu 14.04.5 LTS Server with Dovecot 2.2.9, only local filesystems and authentication done via mysql backend. Server is a Strato Virtual Server running 2.6.32-042stab120.11 #1 SMP Fri Mar 10 16:52:50 MSK 2017 i686 i686 i686 GNU/Linux Sometimes I see errors in the log like this for one user: Error: Transaction log /var/vmail/ example.net/jena/Maildir/.Archives.2017/dovecot.index.log: duplicate transaction log sequence (2) After seeing these, some time later, (mostly 2h), Dovecot starts to deny login from all clients with messages like this: Mar 23 01:32:27 h2312250 dovecot: auth: Error: BUG: Authentication client sent unknown handshake command: REQUEST?1804861441?19318?1?9ce500129289a0237e4e85a6c880a047?session_pid=19323... Mar 23 01:32:27 h2312250 dovecot: imap: Error: Authentication server didn't send valid SPID as expected: MECH#011PLAIN#011plaintext Mar 23 01:32:27 h2312250 dovecot: imap: Error: Disconnected from auth server, aborting (client-pid=19318 client-id=1) Mar 23 01:32:27 h2312250 dovecot: imap-login: Internal login failure (pid=19318 id=1) (internal failure, 1 successful auths): user=< user(a)example.net>, method=PLAIN, rip=X, lip=X, mpid=19323, session=<G9f4AltL4ABRqZ4X> I do not know if these problems have something to do with each other or not. But it happened several times now (maybe once or twice every month or so). When I restart Dovecot, the error seems gone for the moment, until it comes back in a month or some weeks. For reference, here is the running config: dovecot.conf # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab120.11 i686 Ubuntu 14.04.5 LTS auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } plugin { quota = maildir:User quota quota_rule = *:bytes=10M sieve = ~/dovecot.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } ssl_cert = </etc/postfix/sslcert/mailserver.crt ssl_key = </etc/postfix/sslcert/mailserver.key userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol imap { mail_plugins = quota imap_quota } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = sieve quota postmaster_address = postmaster(a)example.net quota_full_tempfail = yes } dovecot-mysql.conf driver = mysql connect = host=localhost dbname=mail user=user password=XXX default_pass_scheme = CRYPT password_query = SELECT password FROM users WHERE email = '%u' user_query = SELECT CONCAT('/var/vmail/',SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1),'/') AS home, CONCAT('maildir:/var/vmail/',SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1),'/Maildir/') AS mail, 5000 AS uid, 5000 AS gid, concat('*:bytes=', quota) AS quota_rule FROM users WHERE email = '%u' Any help is appreciated. Thanks Stefan Koenig

2 2

dovecot configuration oops
by Liam Parker 23 Mar '17

23 Mar '17

Hi, I've just transitioned from using courier to dovecot. Thanks for making this possible. During the 'quite simple' transition process, I didn't do much prep as I'm in more of a more of a fix-compile-fix-compile mode rather than a 'read all the docs first' so you may just want to give up now :) If you are still with me, I just wanted to say, it wasn't completely obvious that I needed a password and a user database [PAM not virtual]. Maybe that's just me but anyway, I spent longer than I should have trying to get PAM to work before eventually realising 'internal error check server logs' meant 'where's your user database'. I guess I assumed PAM could cover it all. If you are in to that kind of thing (usability), it might be sensible to have some kind of "basic errors" check during the config file parse phase which picks up stuff like this and points out a more specific error. I did check the logs vigorously and I even turn on every kind of debug_ (to the extent I started getting plain text passwords in the log!) but the 'internal server error check logs' was actually the only log entry for this issue. In case the point is still unclear, I would have spent less time running in the wrong direction if the error message had been "At least one user database must be configured" or some such. I completely understand if your philosophy is "read the manual or suffer the consequences" and it wasn't a complete fail but a simple up front config check could have helped. Thanks for your time today, Kind Regards, Liam

2 1

Understanding quotas
by Robert Moskowitz 23 Mar '17

23 Mar '17

Just to level-set, I am using Centos7-arm which supplies Dovecot 2.2.10. I don't have access to a arm build or mock environment and use what is available (I DID try to install them but had dependency issues). Much of what I have is from campworld, but some I have picked up from other Centos mailserver builders. Also I am using Postfixadmin which provides quotas per user in mysql database. To access those quotas for dovecot, I have the following script: cat <<EOF>/etc/dovecot/dovecot-dict-quota.conf || exit 1 connect = host=localhost dbname=postfix user=postfix password=$Postfix_Database_Password map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } EOF Where $Postfix_Database_Password is an env variable in my install howto. I am setting up quota plugins as follows: sed -i "/#mail_plugins/ a mail_plugins = quota sieve" /etc/dovecot/conf.d/15-lda.conf sed -i "/#mail_plugins / a mail_plugins = quota imap_quota trash" /etc/dovecot/conf.d/20-imap.conf sed -i "/#mail_plugins/ a mail_plugins = quota" /etc/dovecot/conf.d/20-pop3.conf But I am having 'challenges' with how to actually enable quotas and understanding 90-quota.conf. I am starting from campworld's 'reduced' dovecot.conf seeing: dict { quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } and plugin { quota = dict:user::proxy::quotadict acl = vfile:/etc/dovecot/acls trash = /etc/dovecot/trash.conf sieve_global_path = /home/sieve/globalfilter.sieve sieve = ~/dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /home/sieve/ #sieve_extensions = +notify +imapflags sieve_max_script_size = 1M } I have put the sieve plugin lines into the 90-sieve.conf and the acl into 10-acl.conf. Trash is a separate matter for a later question... So I am looking for guidance as to how to integrate the Postfixadmin quota information properly into 90-quota.conf thank you

2 3

Re: replicator crashing - oom
by Daniel Miller 22 Mar '17

22 Mar '17

Think I got it: #0 0x00007fddaf597c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007fddaf59b028 in __GI_abort () at abort.c:89 #2 0x00007fddaf9c0c86 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:201 #3 0x00007fddaf9c0d6e in i_internal_fatal_handler (ctx=0x7fff7197d000, format=<optimized out>, args=<optimized out>) at failures.c:670 #4 0x00007fddaf958322 in i_panic (format=format@entry=0x7fddafa047b8 "data stack: Out of memory when allocating %lu bytes") at failures.c:275 #5 0x00007fddaf9bee3f in mem_block_alloc (min_size=536870912) at data-stack.c:341 #6 0x00007fddaf9bf08b in t_malloc_real (size=size@entry=536870912, permanent=permanent@entry=true) at data-stack.c:396 #7 0x00007fddaf9bf0ca in t_malloc (size=size@entry=536870912) at data-stack.c:442 #8 0x00007fddaf9da5f8 in pool_data_stack_realloc (pool=<optimized out>, mem=0x7fdd99f59038, old_size=268435449, new_size=536870912) at mempool-datastack.c:126 #9 0x00007fddaf9bcc42 in p_realloc (new_size=<optimized out>, old_size=<optimized out>, mem=<optimized out>, pool=<optimized out>) at mempool.h:88 #10 buffer_alloc (buf=buf@entry=0xf849a8, size=536870912) at buffer.c:36 #11 0x00007fddaf9bcfb4 in buffer_check_limits (data_size=32, pos=268435424, buf=0xf849a8) at buffer.c:75 #12 buffer_write (_buf=0xf849a8, pos=268435424, data=0xfa0420, data_size=32) at buffer.c:187 #13 0x000000000040519b in array_append_i (count=1, data=0xfa0420, array=<synthetic pointer>) at ../../../src/lib/array.h:168 #14 replicator_queue_handle_sync_lookups (user=0xfa3fc0, queue=0xf92260) at replicator-queue.c:278 ---Type <return> to continue, or q <return> to quit--- #15 replicator_queue_push (queue=0xf92260, user=0xfa3fc0) at replicator-queue.c:296 #16 0x00000000004049c2 in dsync_callback (reply=DSYNC_REPLY_OK, state=0xf84590 "AQAAAHX+sC3O3gNSjAoAAOEwx0RQoclMAwAAAAQ", 'A' <repeats 20 times>, "DAAAAEu2cNfSIZk0oMgAAO8QcX0yhyUwDAAAAlg", 'A' <repeats 20 times>, "IAAADFjqMEUPIEU+IlAACdtazLUaHJTAcAAAAI", 'A' <repeats 20 times>, "BwAAAEfYPTqXct9VYRMAAJ21"..., context=0xfa1560) at replicator-brain.c:121 #17 0x00000000004040a8 in dsync_callback (client=0xfa14a0, state=<optimized out>, reply=DSYNC_REPLY_OK) at dsync-client.c:65 #18 0x0000000000404275 in dsync_input_line (line=0xf84988 "+", client=0xfa14a0) at dsync-client.c:132 #19 dsync_input (client=0xfa14a0) at dsync-client.c:153 #20 0x00007fddaf9d47d2 in io_loop_call_io (io=0xfa15a0) at ioloop.c:599 #21 0x00007fddaf9d5d17 in io_loop_handler_run_internal (ioloop=ioloop@entry=0xf8c720) at ioloop-epoll.c:223 #22 0x00007fddaf9d486c in io_loop_handler_run (ioloop=ioloop@entry=0xf8c720) at ioloop.c:648 #23 0x00007fddaf9d4a28 in io_loop_run (ioloop=0xf8c720) at ioloop.c:623 #24 0x00007fddaf961fc3 in master_service_run (service=0xf8c5c0, callback=callback@entry=0x404720 <client_connected>) at master-service.c:641 #25 0x000000000040342a in main (argc=1, argv=0xf8c390) at replicator.c:112 Daniel On 3/22/2017 9:11 AM, Daniel Miller wrote: > Where would I find the core file? I'm not finding anything obvious. > > The replicator path is /usr/local/libexec/dovecot/replicator > > Daniel > > On 3/22/2017 12:52 AM, Aki Tuomi wrote: >> Can you provide us gdb bt full dump? >> >> gdb /usr/libexec/dovecot/replicator /path/to/core >> >> on some systems, it's /usr/lib/dovecot/replicator >> >> Aki >> >> On 21.03.2017 23:48, Daniel Miller wrote: >>> I have the following in my log: >>> >>> Mar 21 14:46:59 bubba dovecot: replicator: Panic: data stack: Out of >>> memory when allocating 1073741864 bytes >>> Mar 21 14:46:59 bubba dovecot: replicator: Error: Raw backtrace: >>> /usr/local/lib/dovecot/libdovecot.so.0(+0x97c90) [0x7f4638a7cc90] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(+0x97d6e) [0x7f4638a7cd6e] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f4638a14322] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(+0x95e3f) [0x7f4638a7ae3f] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(+0x9608b) [0x7f4638a7b08b] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(+0xb15f8) [0x7f4638a965f8] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(+0x93c42) [0x7f4638a78c42] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(buffer_write+0x74) >>> [0x7f4638a78fb4] -> dovecot/replicator(replicator_queue_push+0x13b) >>> [0x40519b] -> dovecot/replicator() [0x4049c2] -> dovecot/replicator() >>> [0x4040a8] -> dovecot/replicator() [0x404275] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) >>> [0x7f4638a907d2] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) >>> >>> [0x7f4638a91d17] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) >>> [0x7f4638a9086c] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>> [0x7f4638a90a28] -> >>> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7f4638a1dfc3] -> dovecot/replicator(main+0x17a) [0x40342a] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>> [0x7f4638640ec5] -> dovecot/replicator() [0x4034c5] >>> Mar 21 14:47:00 bubba dovecot: replicator: Fatal: master: >>> service(replicator): child 15650 killed with signal 6 (core dumped) >>>

1 0

Permission denied when logrotating dovecot.log
by Michael Heuberger 22 Mar '17

22 Mar '17

Hello guys Having headaches here how to make logrotation for dovecot log files work. Having permission issues: michael.heuberger@xxx /e/l/daily ❯❯❯ sudo logrotate -fv dovecot.daily ⏎ reading config file dovecot.daily Handling 1 logs rotating pattern: /var/log/dovecot*.log forced from command line (10 rotations) empty log files are rotated, old logs are removed considering log /var/log/dovecot.log error: skipping "/var/log/dovecot.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation. This is my current logrotation conf for dovecot: /var/log/dovecot*.log { rotate 10 missingok sharedscripts postrotate doveadm log reopen endscript } And the /var/log folder has these permissions: drwxrwxr-x 12 root syslog 4.0K Mar 19 12:43 log Any clues what's wrong? Thanks Michael -- Binary Kitchen Michael Heuberger 1/33 Parrish Road Sandringham Auckland 1025 (New Zealand) Mobile (text only) ... +64 21 261 89 81 Email ................ michael(a)binarykitchen.com Website .............. http://www.binarykitchen.com

5 9

SIS and mailbox restore
by Jean-Luc Oms 22 Mar '17

22 Mar '17

Still on the problem with dovecot 2.2.27 .... I use the single instance storage, and mdbox format. I use 2 zfs file systems, one for mailboxes, the other for attachements. I use znapzend to generate many zfs snapshots for backup purpose (inode numbers are preserved but it is unusefull). When I try to restore a deleted/purged mailbox with doveadm import I get the error of the first attachement missing in the SIS and the process stops. Option 1: the python script is working but it's really too long. * I restore all the SIS missing files since the snapshot date, preserving hard link count; * I restore the mailbox * I purge all the mailbox to delete the large number of unnecessery files restored Option 2: looking in the list archive I see that doveadm dump can guive me the msg.ext-ref that was missing to my knowledge. I try this use of doveadm dump: doveadm dump /MySnapShot/vmail/user/mdbox/storage/m.* Is this correct to get all the information I need ? I have to write a script to get msg.ext-ref for a particular msf.guid, or a specific mailbox Is there a way with doveadm dump to extract this type of information for a particular mailbox ? Thanks for your support. -- __________________________________________ Jean-Luc Oms STI-RéseauX - LIRMM - CNRS/UM2 161 rue Ada - BAT 4 - CC 477 34095 Montpellier cedex 5 Tel +33 4 67 41 85 93 Urg +33 6 32 01 04 17 __________________________________________

1 0

[Sieve] Is the way to run external script to get attachment?
by Konstantin Shalygin 22 Mar '17

22 Mar '17

Hello. Read all about Extprograms, but have a question. Use case: Every day we have couple messages with attachment (from one sender, this is one-way communication). This attachment is uploads to ownCloud. By hand... need some automation. As I see, pipe cat execute scripts with text data USER/FROM/SUBJECT - and I think with variables it can be any text data, how about attachment? How I see this: script executes and as arg receives from sieve attachment file, or message file and we can parse it for attachment. When we have attachment - upload to ownCloud via RESTapi and purge message. Found some realization for encrypt messages https://github.com/EtiennePerot/gpgit/blob/master/encmaildir.sh but for me more easy just connect to IMAP and do what I want (but I love sieve). Thanks. <http://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms>

2 2

Problem with sieve_before
by Ger Hooton Scoil Chroí Íosa Blarney 22 Mar '17

22 Mar '17

Hi All, I cannot get the sieve_before to work. I have dovecot & sieve installed on Debian 7 as follows /etc/dovecot/conf.d/90-sieve.conf : plugin { # The path to the user's main active script. If ManageSieve is used, this the # location of the symbolic link controlled by ManageSieve. sieve = ~/.dovecot.sieve # The default Sieve script when the user has none. This is a path to a global # sieve script file, which gets executed ONLY if user's private Sieve script # doesn't exist. Be sure to pre-compile this script manually using the sievec # command line tool. # --> See sieve_before fore executing scripts before the user's personal # script. sieve_default = /var/lib/dovecot/sieve/default.sieve # Directory for :personal include scripts for the include extension. This # is also where the ManageSieve service stores the user's scripts. sieve_dir = ~/sieve/ # Directory for :global include scripts for the include extension. sieve_global_dir = /var/lib/dovecot/sieve/ # Path to a script file or a directory containing script files that need to be # executed before the user's script. If the path points to a directory, all # the Sieve scripts contained therein (with the proper .sieve extension) are # executed. The order of execution within a directory is determined by the # file names, using a normal 8bit per-character comparison. Multiple script # file or directory paths can be specified by appending an increasing number. sieve_before = /var/lib/dovecot/sieve/before #####sieve_before2 = /var/lib/dovecot/sieve/before/default2.sieve #sieve_before3 = (etc...) # Identical to sieve_before, only the specified scripts are executed after the # user's script (only when keep is still in effect!). Multiple script file or # directory paths can be specified by appending an increasing number. #sieve_after = #sieve_after2 = #sieve_after2 = (etc...) # Which Sieve language extensions are available to users. By default, all # supported extensions are available, except for deprecated extensions or # those that are still under development. Some system administrators may want # to disable certain Sieve extensions or enable those that are not available # by default. This setting can use '+' and '-' to specify differences relative # to the default. For example `sieve_extensions = +imapflags' will enable the # deprecated imapflags extension in addition to all extensions were already # enabled by default. #sieve_extensions = +notify +imapflags ................................. In /var/lib/dovecot/sieve/before I have : drwxrwxrwx 3 mail mail 4096 Mar 20 23:47 .. -rw-rw-rw- 1 root root 195 Mar 21 09:57 default2.svbin -rwxrwxrwx 1 mail mail 157 Mar 21 11:12 default2.sieve default2.sieve is as follows: require ["fileinto"]; if allof (header :matches "subject" "*JUNK*") { fileinto "INBOX.Junk"; } The users personal filters work. Thanks for your help. __________________________________________________ Please consider the environment before printing this email. Ger

3 13