- dovecot - dovecot.org

user configuration question
by Eric Abrahamsen 02 Mar '16

02 Mar '16

I've been using mbsync to sync this email account to a local Dovecot installation. I used to use Gmail, through Google Apps, and a while ago switched the account to self-hosting. I didn't want to upload years' worth of old email to my old server, so I've had two accounts in my MUA: one talking to the local Dovecot account for the old emails, and a second talking directly to the server for the new emails. I'd like to sync the new account to Dovecot as well, and I'm having trouble wrapping my brain around how to configure the two side-by-side accounts in Dovecot (I know there are other solutions to this, but I'm also just trying to figure out how this works). What I don't grok is the exact effect of the different username fields in the passdb file, and how they combine with username_format. Right now my dovecot.conf has this: passdb { driver = passwd-file args = username_format=%u /etc/dovecot/passwd } userdb { driver = static args = uid=eric gid=users home=/home/eric/.mail/%d/%n username_format=%u default_fields = mail=maildir:/home/eric/.mail/%d/%n/mail } And /etc/dovecot/passwd: eric@ericabrahamsen.net:{PLAIN}password:eric@ericabrahamsen.net Say I want to keep my existing mail where it is, and have all the new mail go into ~/.mail/ericabrahamsen.net/eric-new/mail. Ie, when mbsync connects to the local Dovecot, it does so with the username eric-new(a)ericabrahamsen.net. Now, in Dovecot's conf, do I change the email address in the first field of /etc/dovecot/passwd, or the third field, or both? And, if it isn't too much trouble, what is the difference between the two? Thanks! Eric

1 0

Dsync induces redundant mail after rapid append/expunge repeatedly
by 潘信傑 02 Mar '16

02 Mar '16

I meet the problem about dsync with expunge. The problems is expunged mail would reappear after dsyncing and it is easily to reproduce. Environment Settings: - Two servers(serverA, serverB) both enabled plugin "*replication"* - And more detail get by "dovecot -n" show below Reproduce flow: - Run two processes pA and pB both would run in while loop - pA(All behavior is on Draft mailbox and all on serverA): - Clean "Draft" mailbox of userA - Run the following imap command of userA - imap save (save new mail to Draft) - imap delete (delete old mail on Draft) - imap expunge * - Check if draft only have one email (it would be failed in about 50 loops)* - pB(Send mail to serverB): - Send new mail to userA (which would be save in INBOX) It would be easier reproduce by increasing server loading. I have found something - T he expunge changes from server A in dsync connection don't send immediately, these expunge changes appear after 1~2 dsync connection over. - The replicated mail is generated from serverB and sync back to serverA so expunged mail reappeared. dovecot -n (ServerA) # 2.2.19: /etc/dovecot/dovecot.conf # OS: Linux 3.10.77 x86_64 ext4 auth_mechanisms = plain login default_internal_user = root default_login_user = dovecot default_vsz_limit = 0 disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u dsync_target_servers = tcps:10.13.23.139:24245 lda_mailbox_autocreate = yes mail_debug = yes mail_fsync = never mail_location = maildir:/var/spool/mail/%d/%-4.04i/%i/Maildir mail_max_userip_connections = 30 mail_never_cache_fields = * mail_plugins = " notify replication mail_log" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@._/\~&,()[]" mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size flags from subject } postmaster_address = postmaster@localhost protocols = imap pop3 lmtp imap pop3 sieve replication_dsync_parameters = -N -l 30 -U service aggregator { fifo_listener replication-notify-fifo { group = users mode = 0666 user = admin } unix_listener replication-notify { group = users mode = 0666 user = admin } } service doveadm { inet_listener { address = 10.13.23.136,127.0.0.1 port = 24245 ssl = yes } } service imap-login { inet_listener imap { address = 10.13.23.136,127.0.0.1 port = 143 } inet_listener imaps { address = 10.13.23.136,127.0.0.1 port = 993 ssl = yes } process_limit = 256 service_count = 0 } service indexer-worker { process_limit = 8 } service pop3-login { inet_listener pop3 { address = 10.13.23.136,127.0.0.1 port = 110 } inet_listener pop3s { address = 10.13.23.136,127.0.0.1 port = 995 ssl = yes } process_limit = 256 service_count = 0 } service replicator { unix_listener replicator-doveadm { mode = 0666 } } userdb { driver = passwd } protocol lda { mail_plugins = " notify replication mail_log sieve" } dovecot -n (ServerB) $ ./sbin/dovecot -n # 2.2.19: /etc/dovecot/dovecot.conf # OS: Linux 3.10.77 x86_64 auth_mechanisms = plain login default_internal_user = root default_login_user = dovecot default_vsz_limit = 0 disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u dsync_target_servers = tcps:10.13.23.136:24245 lda_mailbox_autocreate = yes mail_debug = yes mail_fsync = always mail_location = maildir:/var/spool/mail/%d/%-4.04i/%i/Maildir mail_max_userip_connections = 30 mail_never_cache_fields = * mail_plugins = " notify replication mail_log" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational reg ex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@._/\~&,()[]" mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size flags from subject } postmaster_address = postmaster@localhost protocols = imap pop3 lmtp imap pop3 sieve replication_dsync_parameters = -N -l 30 -U service aggregator { fifo_listener replication-notify-fifo { group = users mode = 0666 user = admin } unix_listener replication-notify { group = users mode = 0666 user = admin } } service doveadm { inet_listener { address = 10.13.23.139,127.0.0.1 port = 24245 ssl = yes } } service imap-login { inet_listener imap { address = 10.13.23.139,127.0.0.1 port = 143 } inet_listener imaps { address = 10.13.23.139,127.0.0.1 port = 993 ssl = yes } process_limit = 256 service_count = 0 } service indexer-worker { process_limit = 4 } service pop3-login { inet_listener pop3 { address = 10.13.23.139,127.0.0.1 port = 110 } inet_listener pop3s { address = 10.13.23.139,127.0.0.1 port = 995 ssl = yes } process_limit = 256 service_count = 0 } service replicator { unix_listener replicator-doveadm { mode = 0666 } } userdb { driver = passwd } protocol lda { mail_plugins = " notify replication mail_log sieve" } Regards, Jay Pan

2 2

Setup changes 1.x to 2.x -- custom POP3 account handling required.
by Andrew McGlashan 02 Mar '16

02 Mar '16

Hi, Previously I did /special/ processing for some email accounts; if a particular file existed in the user's home directory, then I caused the account to use /var/mail/$USER file for POP -- otherwise it used IMAP. Now, because I adjusted the setup for POP only, IMAP still worked perfectly well. Some users have both POP and IMAP access for various reasons. The special POP processing was done using a shell script [1] in place of: /usr/lib/dovecot/pop3 - which in turn called the normal binary once setting up the environment. Now, I have been forced to upgrade my Debian from squeeze-lts to wheezy and the dovecot setup is quite different. How can I implement my /wrapper/ script for POP handling now? Here is my current setup with dovecot -n ... # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 7.9 mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap pop3" service imap-login { inet_listener imap { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } } ssl = required ssl_cert = </etc/exim4/exim.crt ssl_key = </etc/exim4/exim.key userdb { driver = passwd } [1] This is the content of the original wrapper script: #!/bin/sh if [ -f $HOME/.use-mbox-for-pop3s ]; then export MAIL=mbox:$HOME/mail:INBOX=/var/mail/$USER export USER_INBOX=/var/mail/$USER if [ -f $USER_INBOX ] then export USER_INBOX_GROUP=$(stat -c %G $USER_INBOX) if [ $USER_INBOX_GROUP != $USER ] then ( date ls -l $USER_INBOX chown Debian-exim:$USER $USER_INBOX ls -l $USER_INBOX ) >> /var/log/usr-lib-dovecot--pop3-affinity.log fi fi else if [ -d $HOME/Maildir ]; then export MAIL=Maildir:$HOME/Maildir else export MAIL=mbox:$HOME/mail:INBOX=/var/mail/$USER fi fi # Finally execute the imap/pop3. If you use both, you'll need two scripts. # - this customization just for POP3S ... exec /usr/lib/dovecot/pop3 Thanks AndrewM

2 2

dsync issues due to Prefix difference?
by Mark Little 02 Mar '16

02 Mar '16

Hi everyone, I'm hoping I am just missing something obvious - but I cannot seem to get a one-way migration dsync working between two Dovecot servers because the Prefix configuration is different. I am doing a migration from our old Dovecot servers to some new cPanel based ones. Source: 2.2.15 Dest: 2.2.21 Here is the debug output for the sync: doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so doveadm(testuser(a)example.ca): Debug: Effective uid=1322, gid=1322, home=/khstor3/khmail-mailboxes/l/li/example.ca/testuser doveadm(testuser(a)example.ca): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir doveadm(testuser(a)example.ca): Debug: maildir++: root=/khstor3/khmail-mailboxes/l/li/example.ca/testuser/Maildir, index=, indexpvt=, control=, inbox=/khstor3/khmail-mailboxes/l/li/example.ca/testuser/Maildir, alt= doveadm(testuser(a)example.ca): Debug: Namespace : Using permissions from /khstor3/khmail-mailboxes/l/li/example.ca/testuser/Maildir: mode=0777 gid=default stdin: is not a tty dsync-local(testuser(a)example.ca): Debug: brain M: Local mailbox tree: INBOX guid=f055ac07d79cad508a7a000075e38adc uid_validity=1353555159 uid_next=2085 subs=no last_change=0 last_subs=0 dsync-local(testuser(a)example.ca): Debug: brain M: Local mailbox tree: Junk guid=703a8b1d51ceab56f91e0000667eee40 uid_validity=1391566942 uid_next=1 subs=yes last_change=0 last_subs=1454100049 dsync-local(testuser(a)example.ca): Debug: brain M: Local mailbox tree: Trash guid=9bee9f0f2aacb65039260000b171e44d uid_validity=1353555163 uid_next=25 subs=no last_change=0 last_subs=0 dsync-local(testuser(a)example.ca): Debug: brain M: Local mailbox tree: Drafts guid=4d03661058a0f1524f240000e9d81ad0 uid_validity=1391566936 uid_next=1 subs=no last_change=0 last_subs=0 dsync-local(testuser(a)example.ca): Debug: brain M: Local mailbox tree: Sent guid=99ee9f0f2aacb65039260000b171e44d uid_validity=1353555161 uid_next=4 subs=no last_change=0 last_subs=0 dsync-local(testuser(a)example.ca): Debug: brain M: Remote mailbox tree: INBOX guid=6460141741cfab5643470000afb0d608 uid_validity=1454100291 uid_next=2064 subs=no last_change=0 last_subs=0 dsync-local(testuser(a)example.ca): Debug: brain M: Remote mailbox tree: INBOX.Trash guid=6360141741cfab5643470000afb0d608 uid_validity=1454100290 uid_next=12 subs=yes last_change=0 last_subs=1454100289 dsync-local(testuser(a)example.ca): Debug: brain M: Remote mailbox tree: INBOX.Sent guid=6260141741cfab5643470000afb0d608 uid_validity=1454100289 uid_next=2 subs=yes last_change=0 last_subs=1454100289 dsync-remote(testuser(a)example.ca): Error: Couldn't find namespace for mailbox Junk I believe the problem is because on the source we have no prefix set, but on the destination it has INBOX as the prefix. I need to use dsync because we have many POP3 clients so I need to ensure the UIDL's are copied correctly. Here is the output from 'doveconf namespace': Source: namespace inbox { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe driver = special_use = \Drafts } mailbox Junk { auto = subscribe driver = special_use = \Junk } mailbox Sent { auto = subscribe driver = special_use = \Sent } mailbox "Sent Messages" { auto = no driver = special_use = \Sent } mailbox Trash { auto = subscribe driver = special_use = \Trash } order = 0 prefix = separator = subscriptions = yes type = private } Destination: namespace { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Archive { auto = create autoexpunge = 0 comment = driver = special_use = \Archive } mailbox Archives { auto = no autoexpunge = 0 comment = driver = special_use = \Archive } mailbox Drafts { auto = subscribe autoexpunge = 0 comment = driver = special_use = \Drafts } mailbox Junk { auto = create autoexpunge = 0 comment = driver = special_use = \Junk } mailbox Sent { auto = subscribe autoexpunge = 0 comment = driver = special_use = \Sent } mailbox "Sent Messages" { auto = no autoexpunge = 0 comment = driver = special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 0 comment = driver = special_use = \Trash } order = 0 prefix = INBOX. separator = subscriptions = yes type = private } If anyone could provide some assistance it would be greatly appreciated! Thanks, Mark

2 6

where is pop3_migration_ignore_missing_uidls=yes set?
by Leon Kyneur 02 Mar '16

02 Mar '16

While doing pop3 migrations with dsync I am getting an error: dsync(user(a)domain.net): Error: pop3_migration: 2 POP3 messages have no matching IMAP messages (first POP3 msg 2638 UIDL fb3d6f3299541e695a28585f5803ed1c) - set pop3_migration_ignore_missing_uidls=yes to continue anyway dsync(user(a)domain.net): Error: Exporting mailbox INBOX failed: Can't lookup pop3-uidl for UID=3252: POP3 UIDLs couldn't be synced Can't see this is doveconf -a so it's not a setting? - tried configuring it as a plugin setting but that didn't work... Where is the correct place to set this? TIA

2 1

Alternative storage backend for Dovecot
by Oleksandr Natalenko 01 Mar '16

01 Mar '16

Hi. We have corporate Dovecot setup with GlusterFS as file storage. We store maildirs on GlueterFS share, mounting it via NFS. Mounting share via native GlusterFS FUSE client introduces visible performance overhead. Also, NFS mount does not provide us with HA. That's why I have an idea to use GlusterFS C API to access maildirs share. I'd like to preserve maildir structure, but get access to it not via FS mount point but with GlusterFS C API calls. I've discovered that Dovecot source tree has src/lib-fs infrastructure with fs-posix implemented on top of it. Is that the right place to start from? GlusterFS C API calls are POSIX-like, so they could be easily used as substitutions for VFS ops. Could such a backend be implemented as separate plugin? Are there any examples of alternative storage backend to look at? Thanks. Regards, Oleksandr.

2 1

search problem dovecot 2.2.21 + fts - Solr
by Anderson Barbosa 01 Mar '16

01 Mar '16

Hello, Realized update dovecot on my server. Now the search is returning differently from the previous version bringing reference information of other messages . For example when doing a search for anderson.joao this new version of the dovecot dovecot 2.2.21 + fts - Solr response will be all email that has the word anderson and joao, instead of returning only items with the word anderson.joao. Before used version 2.2.18 + dovecot fts - Solr and the problem did not occur . For example practical test : Dovecot 2.2.18 # telnet SERVER 143 Trying SERVER... Connected to SERVER. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Zimbra IMAP4. a login conta(a)conta.com.br 1223456 a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAn a select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 14 EXISTS * 0 RECENT * OK [UIDVALIDITY 1452548222] UIDs valid * OK [UIDNEXT 25] Predicted next UID * OK [HIGHESTMODSEQ 52] Highest a OK [READ-WRITE] Select completed (0.001 secs). a SEARCH text "anderson" * SEARCH 11 12 (2 found emails) a OK Search completed (0.265 secs). a SEARCH text "joao" * SEARCH 13 14 (2 found emails) a OK Search completed (0.003 secs). a SEARCH text "anderson.joao" * SEARCH (0 found emails) a OK Search completed (0.004 secs). a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. Dovecot 2.2.21 # telnet SERVER 143 Trying SERVER... Connected to SERVER. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Zimbra IMAP4. a login conta(a)conta.com.br 1223456 a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAn a select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 14 EXISTS * 0 RECENT * OK [UIDVALIDITY 1452548222] UIDs valid * OK [UIDNEXT 25] Predicted next UID * OK [HIGHESTMODSEQ 52] Highest a OK [READ-WRITE] Select completed (0.000 + 0.000 secs). a SEARCH text "anderson" * SEARCH 11 12 (2 found emails) a OK Search completed (0.004 + 0.000 secs). a SEARCH text "joao" (2 found emails) * SEARCH 13 14 a OK Search completed (0.005 + 0.000 secs). a SEARCH text "anderson.joao" * SEARCH 11 12 13 14 *(4 found emails)* a OK Search completed (0.005 + 0.000 secs). a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. Even using characters Special "" \ scape, ' ' for an answer will always be all emails with the word anderson and joao. Checking the Changelog dovecot noticed que NAS versions Previous v2.2.20 and v2.2.19 certain changes with respect to fts . There Have Another way to Make Search for Exact Word In this new version to loft? -- Anderson Barbosa dos Santos Em todo o universo, nada existe de mais parecido com Deus que o silêncio. "Johanes Eckhart"

3 3

Inconsistent mdbox index while running doveadm altmove
by Ricardo Machini Barbosa 01 Mar '16

01 Mar '16

Hello, I was performing doveadm altmove for one account while this mailbox was being accessed. Dovecot version: 2.2.21 Mailbox format: mdbox Command: doveadm altmove -u user1(a)domain.com.br before 360d This command took about 5 minutes to finish and during the execution I found some log messages: Feb 25 19:57:51 dovecot: imap(user1(a)domain.com.br): Error: mdbox map /var/mail/user1/mdbox/storage/dovecot.map.index corrupted: Unexpectedly lost INBOX uid=77 map_uid=76 Feb 25 19:57:51 dovecot: imap(user1(a)domain.com.br): Internal error occurred. Refer to server log for more information. [2016-02-25 19:57:51] in=4798 out=1361819 deleted=0 expunged=0 trashed=0 Feb 25 19:57:51 dovecot: imap(user1(a)domain.com.br): Warning: fscking index file /var/mail/user1/mdbox/storage/dovecot.map.index Feb 25 19:58:15 dovecot: imap(user1(a)domain.com.br): Error: /var/mail/user1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Feb 25 19:58:20 dovecot: imap(user1(a)domain.com.br): Error: /var/mail/user1/mdbox/mailboxes/Rascunhos/dbox-Mails/dovecot.index reset, view is now inconsistent After that these messages moved to alternate storage disappeared from the mailbox/IMAP but these files were all on alternative storage. I had to execute doveadm force-resync -u user1(a)domain.com.br INBOX to recovery this mailbox, but losing all e-mail's flags. I should do some kind of lock before running the command altmove ? Regards, Ricardo

2 1

Re: severe fork() problems with new dovecot server
by J. Niklas 01 Mar '16

01 Mar '16

On 01.03.2016 18:21, Dolf Schimmel wrote: > Recently I played around a little with cgroups where you can limit the max number of processes per cgroup. > Could it be that, perhaps, you've stumbled upon such a limit? > Systemd does contain all services by default in their own cgroup afaik, > so it could be that you're using it unknowingly. Yes, yes, \o/ ;-) #> systemctl status dovecot.service * dovecot.service - Dovecot IMAP/POP3 email server Loaded: loaded (/lib/systemd/system/dovecot.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/dovecot.service.d `-ulimits.conf Active: active (running) since Tue 2016-03-01 15:28:29; 4h 24min ago Main PID: 10098 (dovecot) Tasks: 204 (limit: 512) CGroup: /system.slice/dovecot.service There ist my "512". The way of systemd, ignoring all the config stuff that has been there for decades while inposing its own, complex and very sparsely documented ruleset on top is beginning to seriously annoy me. At least I would have expected some sort of syslog message. Just for the records, this can be changed by adding e.g. TasksMax=4096 to the /etc/systemd/system/dovecot.service.d/ulimits.conf I cited in my OP. Now I'll have to wait and see how things will evolve tomorrow. Thank you so much for the great hint! -- kind regards, Jon

2 1

severe fork() problems with new dovecot server
by J. Niklas 01 Mar '16

01 Mar '16

Hi all, I am getting to you because I am pretty desperate by now. Please read this and share your ideas. I set up a new Mail server with roughly 750 active accounts on it, based on the usual dovecot+postfix+amavis+sql based vmail setup. This is not the first time I did something like this, but admittedly my largest setup so far. I took my time to setup everything pretty straight and stable and migrated all accounts in time with almost no glitches using imapc and pop3c. Things started out smoothly - until the first business morning when the log suddenly began to pop messages like these: dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable dovecot: master: Error: service(pop3): fork() failed: Resource temporarily unavailable The error is not confined to a certain service. There are no other messages in dmesg or syslog which could help in closing in on the root cause. The server always runs at load around 0.4 to 0.5 and has plenty of RAM and Disk, too. Of course I checked resource limits and cranked up everything I could possibly think of to numbers that should easily suffice: /etc/security/limits.conf: daemon soft nproc 2048 daemon hard nproc 3128 dovecot soft nproc 2048 dovecot hard nproc 3128 vmail soft nproc 2048 vmail hard nproc 3128 vmail soft nofile 32768 vmail hard nofile 49152 /etc/systemd/system/dovecot.service.d/ulimits.conf: [Service] LimitNOFILE=65534 However, all my attempts on resolving the problem were futile so far. To work around the problem I implemented a log watcher which constantly looks for the fork failure and dumps some debug info before restarting dovecot to make the problem "go away" - until dovecot hits whatever barrier there is again. Here are today's numbers I gathered from this: > forkdump_03-01_09-42: pop=3, imap=446, all=503, postfix:22, sys: 716 > forkdump_03-01_10-09: pop=9, imap=444, all=510, postfix:28, sys: 741 > forkdump_03-01_10-42: pop=5, imap=450, all=507, postfix:25, sys: 748 > forkdump_03-01_11-18: pop=5, imap=451, all=512, postfix:29, sys: 757 > forkdump_03-01_11-48: pop=3, imap=474, all=511, postfix:37, sys: 765 > forkdump_03-01_12-23: pop=7, imap=444, all=511, postfix:22, sys: 752 > forkdump_03-01_12-53: pop=3, imap=444, all=503, postfix:24, sys: 747 > forkdump_03-01_13-21: pop=3, imap=458, all=504, postfix:31, sys: 766 > forkdump_03-01_13-41: pop=4, imap=451, all=505, postfix:23, sys: 755 > forkdump_03-01_14-35: pop=5, imap=436, all=502, postfix:31, sys: 768 > forkdump_03-01_14-59: pop=3, imap=445, all=501, postfix:27, sys: 764 > forkdump_03-01_15-28: pop=6, imap=452, all=507, postfix:25, sys: 769 There you see the number of pop3 and imap childs in the process list as well as the total number of all dovecot processes and all postfix processes, summarized directly after the fork problem appears in the log. The last column is the sum of all processes in the system. Here is a run where the numbers are measured as the server approaches the last entry in the list above. > 15:26:15: pop=3, imap=432, all=485, VSZ: 14353k, postfix:30, sys: 749 > 15:26:30: pop=4, imap=435, all=490, VSZ: 14497k, postfix:30, sys: 754 > 15:26:45: pop=3, imap=439, all=492, VSZ: 14560k, postfix:28, sys: 754 > 15:27:00: pop=3, imap=444, all=497, VSZ: 14711k, postfix:28, sys: 760 > 15:27:15: pop=4, imap=442, all=496, VSZ: 14673k, postfix:25, sys: 756 > 15:27:30: pop=4, imap=444, all=497, VSZ: 14721k, postfix:25, sys: 757 > 15:27:46: pop=3, imap=446, all=498, VSZ: 14756k, postfix:25, sys: 758 > 15:28:01: pop=3, imap=446, all=497, VSZ: 14718k, postfix:24, sys: 756 > 15:28:16: pop=3, imap=455, all=507, VSZ: 15018k, postfix:25, sys: 767 > 15:28:18: pop=3, imap=455, all=507, VSZ: 15018k, postfix:25, sys: 767 > 15:28:20: pop=3, imap=452, all=504, VSZ: 14931k, postfix:25, sys: 764 ---- service(pop3): fork() failed: Resource temporarily unavailable ---- > 15:28:22: pop=4, imap=452, all=505, VSZ: 14954k, postfix:25, sys: 770 > 15:28:24: pop=4, imap=452, all=505, VSZ: 14954k, postfix:25, sys: 770 > 15:28:26: pop=4, imap=451, all=504, VSZ: 14925k, postfix:25, sys: 769 > 15:28:28: pop=5, imap=451, all=506, VSZ: 14980k, postfix:25, sys: 771 ---- dovecot restart complete ---- > 15:28:30: pop=3, imap=11, all=22, VSZ: 485k, postfix:25, sys: 282 > 15:29:00: pop=4, imap=65, all=83, VSZ: 2293k, postfix:26, sys: 344 > 15:29:31: pop=3, imap=91, all=111, VSZ: 3129k, postfix:27, sys: 373 It always seems to happen shortly after the number of dovecot processes goes beyond 500. The VSZ column is the VSZ sum of all dovecot procs. The number of files opened by vmail (lsof -u <vmailuid> | wc -l) at that moment usually is between 13k and 15k. sysctl fs.file-nr mostly is around 16k total but the system limit is set way higher (at 1532853) However, as you can see in the following dump, there is no process_limit or whatever being set to 500 or 512 or similar. # --------- su vmail -c "ulimit -Ha:" --------------- core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 60026 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 49152 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 3128 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited # ---------------- dovecot -n (mildly anonymized) --------------- # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 4.4.0-8-generic x86_64 Ubuntu Xenial Xerus (development branch) xfs auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_mechanisms = plain login default_client_limit = 10000 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } hostname = my.mail.host imap_client_workarounds = delay-newmail lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To listen = * mail_location = sdbox:/mailsrv/users/%d/%u mail_plugins = quota notify mail_log fts managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts_autoindex = yes quota = dict:User quota::proxy::quota sieve = file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30s stats_track_cmds = yes } postmaster_address = postmaster(a)my.mail.host protocols = " imap lmtp sieve pop3" service anvil { client_limit = 11227 } service auth { client_limit = 11324 unix_listener /mailsrv/spool/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { process_limit = 10000 process_min_avail = 5 service_count = 0 vsz_limit = 2 G } service imap { process_limit = 2048 vsz_limit = 2 G } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { process_limit = 1024 process_min_avail = 3 service_count = 0 vsz_limit = 2 G } service pop3 { process_limit = 1024 } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert = </etc/ssl/local/my.mail.host.crt.pem ssl_key = </etc/ssl/local/my.mail.host.key.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = quota notify mail_log fts quota sieve } protocol lda { mail_plugins = quota notify mail_log fts sieve } protocol imap { mail_max_userip_connections = 15 mail_plugins = quota notify mail_log fts imap_quota stats imap_stats } protocol pop3 { mail_max_userip_connections = 15 } # ------------ end dovecot -n ------------- So, here I am, out of ideas for the moment. Do you have any? -- kind regards, Jonathan

1 0

What happened to dovecot.fi?
by Pieter Koorts 01 Mar '16

01 Mar '16

Hi All, Any news about what happened to dovecot.fi ? Is enterprise or the S3 backend plugin still available? Reason for asking is every attempt to access the dovecot.fi website shows it as down and has done for a while. Pieter

2 1

Returning custom autoexpunge for user's INBOX
by Tom Johnson 01 Mar '16

01 Mar '16

I want to be able to have custom autoexpunge settings for different users. I have it working for various folders (draft, junk, trash), but I can't seem to get it working for the user's primary inbox. In my user_query (I'm using mysql), I'm returning entries like this in the userdb_import field: namespace/inbox/mailbox/autoexpunge=5d namespace/inbox/mailbox/Drafts/autoexpunge=5d namespace/inbox/mailbox/Junk/autoexpunge=5d namespace/inbox/mailbox/Trash/autoexpunge=5d And everything works fine for Drafts, Junk, and Trash, but not for the main inbox. The logs show: Debug: Unknown userdb setting: plugin/namespace/inbox/mailbox/autoexpunge=5d I've also tried variations, trying to get the namespace correct for the main inbox, but they all get the same "Unknown userdb setting" error: namespace/inbox/mailbox/autoexpunge=5d namespace/inbox/mailbox/inbox/autoexpunge=5d namespace/inbox/mailbox/Inbox/autoexpunge=5d namespace/inbox/mailbox/INBOX/autoexpunge=5d I suspect I'm just giving the wrong namespace for the primary inbox. What's the right way to do this? Tom

2 2

UnicodeData
by Vincent Dahmen 29 Feb '16

29 Feb '16

Hey guys, there seem to be a problem with wget and/or the unicode webserver. During the make process there are several files wich are download from the unicode server. I can access them manually but both curl and wget failing to get it. Are they any other mirrors or anyone else expiring this bug? Greetings

2 3

Re: sieve_extprograms - double linebreaks at filtering
by Hajo Locke 29 Feb '16

29 Feb '16

Hello, Am 09.09.2015 um 13:30 schrieb Stephan Bosch: > > > Op 9-9-2015 om 12:37 schreef Hajo Locke: >> Hello, >> >> >> Am 09.09.2015 um 11:26 schrieb Stephan Bosch: >>> >>> >>> Op 8-9-2015 om 13:10 schreef Hajo Locke: >>>> Hello List, >>>> >>>> i have a problem when using sieve-plugin sieve_extprograms. I use >>>> dovecot 2.2.18 and bundled pigeonhole 0.4.6 (Ubuntu 14.04.3 LTS) >>>> >>>> i have enabled sieve_extprograms and vnd.dovecot.filter to send >>>> mail to user-defined script and get changed content back. >>>> My script previously was used with procmail and is working fine. >>>> Using same script with vnd.dovecot.filter leads to odd behavior. >>>> I already found out the problem itself: >>>> By sending Mailcontent to filter-programm it seems that >>>> sieve/sieve_extprograms is adding additional linebreaks to every >>>> line of complete mail. >>>> Please see this image to clarify: >>>> http://r31i.imgup.net/header8d56.jpg?l=de >>>> Every lineending got additional windows-lineendings, which leads to >>>> problems with processing scripts. >>>> Piping mails by procmail to same script is working without >>>> problems, because this mails are recieved "clean" without the "^M". >>> >>> Well, the specified format for an internet message like e-mail has >>> CRLF line endings everywhere. That's why this happens now: >>> >>> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/file/5df1b6d72ec2/src/plugins… >>> >>> >>> But maybe that is not such a good idea in a UNIX environment. I >>> think I can just make it configurable. >> >> ahh, there it is. i think just commenting out this line in my sources >> will result in some errors. Is there a possibility for me to make a >> quickfix in my sources? (use other function or something like that) > > You can replace it with the i_stream_create_lf() function, which > enforces LF rather than CR+LF, which would be what you need. Unfortunately i have to continue this issue. During my last updates this problem is occured again. This are my versions: # 2.2.21 (): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.11 (6ddf62b87f60+) # OS: Linux 3.13.0-77-generic x86_64 Ubuntu 14.04.4 LTS I tried to configure the linebreaks in my plugin section of /etc/dovecot/dovecot.conf by: sieve_extprograms_input_eol = lf But it seems to have no effect. Is this configured the right way or i have to use 0.4.12? Currently it looks like this: plugin { sieve = ~/sieve/.dovecot.sieve sieve_before = ~/sieve_before/.dovecot_before.sieve sieve_after = ~/sieve_after/.dovecot_after.sieve sieve_plugins = sieve_extprograms sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.filter +vnd.dovecot.execute +editheader sieve_pipe_bin_dir = /usr/local/bin/ sieve_filter_bin_dir = /usr/local/bin/ sieve_dir = ~/sieve sieve_global_dir = /etc/dovecot/sieve/ sieve_extprograms_input_eol = lf } > > Regards, > > Stephan. > > Thanks, Hajo

2 2

Queries are not being sent to solr when search on server issued via imap client.
by ralph.jones＠modolingo.de 29 Feb '16

29 Feb '16

I traced the network communication with the solr server with tcpdump: sudo tcpdump -i any -A -vvv port 8080 I logged into the imap server via telnet (actually openssl) and issued the command ". search body <string>". About 20 results were returned within about half a second. Nothing was seen in tcpdump. When I issue "doveadm search box <string>", I can see the search string in the output of tcpdump and also in the solr logs. Solr seems to have been initialized correctly, in that when I issue a query via curl, I receive the correct results. I am using: dovecot 2.2.13 (debian jessie) libsolr-java 3.6.2+dfsg-5 relevant config: mail_plugins = fts fts_solr acl virtual plugin { fts = solr fts_solr = url=http://localhost:8080/solr/ debug ... }

2 1

Dovecot sieve - How to check which username a mail is being delivered to?
by Yannik Sembritzki 27 Feb '16

27 Feb '16

Hi everyone, I am looking for a way to check which mailbox an email is being delivered to in a global sieve filter. After not being able to find some kind of variable that is populated by dovecot automatically, I tried to use the `Delivered-To` header. I tried a check like this: header :is "delivered-to" "<user(a)domain.org>" Unfortunately, this does not only match the first occurence of the header (which can be assumed is added by dovecot), but any occurence of this header in the email, which could be added due to previous deliveries and forwards, or just cause of malice. Is there any way to achieve this? Best regards, Yannik

2 3

Assertion crash: (idx * array->element_size < array->buffer->used)
by Joseph Tam 27 Feb '16

27 Feb '16

Log message: Traceback is hex gibberish: if symbolic traceback is needed, I'll recompile with "cc -g" and retest. Panic: file ../../../src/lib/array.h: line 219 (array_idx_i): assertion failed : (idx * array->element_size < array->buffer->used) (Similar to http://dovecot.org/pipermail/dovecot/2015-February/099771.html) IMAP command that provoked the crash: x1 FETCH 3 (BODY) Dovecot version: 2.2.19, Solaris10 (I don't think doveconf -n is relevant, but willing to provide if needed). It seems like a mailbox index issue. Removing, then recreating the indices gets rid of the crash. I saved these indices if it will help. Joseph Tam <jtam.home(a)gmail.com>

1 0

Different Quotas For Public Namespace
by Chris 26 Feb '16

26 Feb '16

Dear All, are different quotas for public namespace folders possible? They're in the same namespace. If I got it right, the wiki says there's one quota per namespace only? - Chris

2 2

Block public namespace mail when quota exceeded
by Chris 26 Feb '16

26 Feb '16

Dear All, is it possible to temporarily reject mails, when quota of public namespace subfolder is exceeded? I'd check folder size with a cronjob. Dovecot is using a system user, so I guess blocking post and insert in ACL isn't a solution? I don't want to set the folder to read-only, that users can still delete mails. Thank you in advance. - Chris

2 2

Question: Directory hashing to support large domains | Dovecot & MySQL
by Altin Ukshini 26 Feb '16

26 Feb '16

Hello there, I have a postfix + dovecot + mysql installation in an ubuntu server. Virtual users and domains in a small mysql db that I manage via a web based mailadmin I wrote myself. The mysql db I'm talking about has a qmail structure that used to be managed by qmailadmin and worked with vpopmail. As for information, I cannot move to a different structure (exc postfixadmin/postfix) for many reasons, so I was forced to stick with it and configure postfix and dovecot to read that structure... If you may know, qmail (depends on the configuration) uses a directory hashing like /home/vpopmail/DOMAIN/P/7/USERNAME (something similar) in order to manage large domains, and this is handled by vpopmail. Please refer to these two links for details: - http://free.acrconsulting.co.uk/email/qmh.html - https://sourceforge.net/p/vpopmail/mailman/message/2710408/ I saw that dovecot uses something similar which I found in dovecot documentation : http://wiki2.dovecot.org/MailLocation#Directory_hashing Unfortunately, I cannot find more details, so I'd be thankful if you help me out with the following: By reading the documentation, I can tell that I can use the dir hashing variables in my dovecot configuration. But, in my case, I get the directory path from my mysql db via dovecot sql user_query - not from mail_location variable in 10-mail.conf Therefore if in case of a first received email for a user: dovecot checks in the db, gets the path and creates the maildir from the given path. In case the maildir column is empty, it fails, and it also fails to authenticate... So, can dovecot write to mysql? I would like to use directory hashing of dovecot, but from what I can find: If dovecot has mail_location variable set to do the dir hashing, it will not then go and add the created path in my mysql db. While if I try to implement something similar from my web gui application, I'd probably have to rewrite the algorithm that does this!? Can anyone help me figure out what's the best way to implement this? Thank you very much in advance for your help! Best, Altin Ukshini

1 0

[Feature Request] doveadm option to return number of messages acted upon
by Haravikk 26 Feb '16

26 Feb '16

So I have a script for handling my specific archive and expunge needs, but it’d be nice to be able to track how many messages are being affected. Currently I’m doing it by firing the same search queries into doveadm search and counting the lines of the result with wc -l, but that’s not a very pretty solution. While I’m mostly doing it out of interest on a personal server, I can’t imagine it’s a very scalable way to do it if you wanted to gather some kind of metrics for example. What I think would make sense would be for relevant doveadm commands such as move, expunge (probably purge too) and others I haven’t thought of to have a new option that, if enabled, will cause the command to output the number of messages affected as the final line of output (most of these commands don’t have any output anyway). A doveadm count command could also be a convenient, more efficient, alternative to doveadm search | wc -l.

2 1

Replication between two ISPCOnfig servers
by Otto Saayman 26 Feb '16

26 Feb '16

Hello I have set up two servers, one master and one slave, using ISPConfig. I have been using rsync to synchronise mails between the two machines, but this has some unstable results. I had a look at setting up replication: http://wiki.dovecot.org/Replication When I run the command, doveadm user '*', only two users are returned. So when I run this command: doveadm replicator replicate \* I get: 2 users updated I run this command: doveadm sync -u [e-mail address] tcp:[2nd/ slave dovecot server IP address] Here I can input any valid e-mail address in the ISPConfig MySQL database and it works. How do I tell the dovecote server to replicate all the users configured in the MySQL database it uses when it handles logins via POP/ IMAP, all of which is working fine? Is there a command I need to run at a set interval via cron to synchronise the two machines' mailboxes, which are identical in every way except the mails the users' mailboxes contain? OR Is there a way that I can instruct the two machines to keep each other's user mailbox contents "up to date"? The main idea is to have a hot backup of my "main" dovecot server and also to enable a lower mail priority for the second server so that mails are still received when the main server becomes busy, or is taken offline for some reason. Any help, or link to a HOWTO would be appreciated. -- Best Regards Otto Saayman

2 1

Checking index sanity
by Emmanuel Dreyfus 25 Feb '16

25 Feb '16

Hello We experienced corrupted dovecot indexes, probably caused by a server crash. This does not cause a lot of harm, just annoyance. For instance, LMTP raises an assertion during delivery, which causes mail to multiple recipients e-mail to remain in the queue if a single of them hits a corrupted index. The workaround for now is to detect the situation in the logs and to remove corrupted indexes when the problem arise. A better fix would be to sanity check all user's index on startup. Is there a command line tool to do this? -- Emmanuel Dreyfus manu(a)netbsd.org

1 0

VS: Re: SQLite driver and auth-worker credentials
by Aki Tuomi 24 Feb '16

24 Feb '16

this is probably something that ought to be fixed. Read only scenario for auth/userdb is quite plausible. ---Aki TuomiDovecot oy-------- Alkuperäinen viesti --------Lähettäjä: Lev Serebryakov <lev(a)serebryakov.spb.ru> Päivämäärä: 24.2.2016 23.32 (GMT+02:00) Saaja: james(a)lottspot.com Kopio: dovecot(a)dovecot.org Aihe: Re: SQLite driver and auth-worker credentials Hello James, Wednesday, February 24, 2016, 11:17:12 PM, you wrote: >> But system should assign all secondary GIDs to effective UID? > separate 'group =' directive in addition to the 'user =' directive > (http://wiki2.dovecot.org/Services) Oh, I missed this one, thank you -- Best regards, Lev mailto:lev@serebryakov.spb.ru

1 0

SQLite driver and auth-worker credentials
by Lev Serebryakov 24 Feb '16

24 Feb '16

I want to use SQLite database as storage for auth and user databases. I've encountered two problems here: (1) There is no way to open SQLite database read-only (via sqlite3_open_v2() call with SQLITE_OPEN_READONLY flag). It looks bad. I don't need (and want) to give dovecot rights to write to this database. (2) I've created system group "hostingdb", added "dovecot" user to it and gives 660 rights to database file, but still "auth-worker" could not open database and complains to log file. Now I'm set "user = root" for auth-worker, but I don't like it! Why auth-worker doesn't belong to "hostingdb" group? -- // Black Lion AKA Lev Serebryakov

2 4