From dovec at nettrust.co.nz Sat Sep 1 01:42:32 2012 From: dovec at nettrust.co.nz (Michael) Date: Sat, 01 Sep 2012 10:42:32 +1200 Subject: [Dovecot] dovecotadm error Message-ID: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> When I use the following command: doveadm expunge -A mailbox Junk savedbefore 28d I get: doveadm(root): Error: User listing returned failure In the mail log file it says: dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table 'system.users' doesn't exist (using built-in default iterate_query: SELECT username, domain FROM users) However this is setup: dovecot-sql.conf: iterate_query = SELECT username AS user FROM accounts though I have had to comment the following from dovecot.conf: #iteratedb { # args = /usr/local/etc/dovecot/dovecot-sql.conf # driver = sql #} because it errors on dovecot start up. I have already spent a bit of time on the Dovecot website and Google trying to find an answer. I want to get the expires plugin working. Any leads please? dovecot -n # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.29.6-rt24-smp i686 Slackware 13.1.0 auth_master_user_separator = * auth_mechanisms = plain login dict { expire = mysql:/usr/local/etc/dovecot/dovecot-dict-expire.conf quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 mail_access_groups = dovecot mail_location = Maildir:%h/Maildir mail_plugins = " expire" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Drafts autocreate2 = Junk autocreate3 = Sent autocreate4 = Trash autosubscribe = Drafts autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Trash expire = Junk expire_dict = proxy::expire quota = dict:user::proxy::quotadict sieve = %h/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/before.sieve sieve_dir = %h/sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service dict { unix_listener dict { group = dovecot mode = 0660 } } service imap-login { inet_listener imap { address = DELETED } } service managesieve-login { executable = /usr/local/libexec/dovecot/managesieve-login } service managesieve { executable = /usr/local/libexec/dovecot/managesieve } ssl_cert = I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) disconnect in IDLE and inactivity time outs. I'm using only IMAP with two users. Both users access the server from multiple devices, phone, tablet and desktop computer. I had previously set the max_userip_connections to 20 and rarely got the following limits exceeded message. That's probably being triggered when we are both home and using all of our devices on the same network. I've now changed it to 40 and we'll see how that does. The other is the IDLE and inactivity timeouts. I don't really understand how IDLE works and I couldn't find an inactivity timeout setting to play with. There are at this second 32 imap or imap-login processes running with only two phones and one desktop running. That's a few more than before the upgrade but not a lot. --------------------- Dovecot Begin ------------------------ Dovecot limits exceeded: max_userip_connections: penny from 216.240.58.138 to 216.240.58.140: 56 Time(s) Dovecot disconnects: Disconnected in IDLE: 485 Time(s) Inactivity: 343 Time(s) no auth attempts: 14 Time(s) no reason: 21 Time(s) Everything seems to be working OK so maybe I should just not worry about it? Thanks, knute... # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_anonymous_username = anonymous auth_cache_negative_ttl = 1 hours auth_cache_size = 0 auth_cache_ttl = 1 hours auth_debug = no auth_debug_passwords = no auth_default_realm = auth_failure_delay = 2 secs auth_first_valid_uid = 500 auth_gssapi_hostname = auth_krb5_keytab = auth_last_valid_uid = 0 auth_master_user_separator = auth_mechanisms = plain login auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 60 default_internal_user = dovecot default_login_user = dovenull default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins disable_plaintext_auth = yes dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_proxy_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 first_valid_gid = 1 first_valid_uid = 500 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = imap_idle_notify_interval = 2 mins imap_logout_format = bytes=%i/%o imap_max_line_length = 64 k import_environment = TZ info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/lib/dovecot listen = *, :: lmtp_proxy = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = syslog log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = Dovecot ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib/dovecot/modules mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no passdb { args = deny = no driver = pam master = no pass = no } pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = " imap" quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs inet_listener { address = port = 0 ssl = no } privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener director-userdb { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 0 ssl = no } inet_listener pop3s { address = port = 0 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = Hi, I've not found the answer to this question anywhere - please forgive me if I overlooked. I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net From acrow at integrafin.co.uk Sat Sep 1 12:42:16 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Sat, 01 Sep 2012 10:42:16 +0100 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: <5041D878.3090609@integrafin.co.uk> On 01/09/12 09:10, David Anderson wrote: > Hi, > > I've not found the answer to this question anywhere - please forgive > me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are > hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same > way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and > logwatch can be noisy and it's easy to overlook. > > My motivation: for some reason my POP3 listener was hanging, and I had > to restart dovecot (this is 2.0.9). But I was not aware - the process > was still there, it was accepting connections (but not doing anything > useful with them) and was actually managing to log the fact that the > process limit was being hit (as more and more connections came in). If > I could have been alerted I could have restarted dovecot earlier. > > Many thanks, > David > I believe that nagios or icinga could do this for you with a log analyser plugin. http://exchange.nagios.org/directory/Plugins/Log-Files Cheers Alex From leamhall at gmail.com Sat Sep 1 12:53:15 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 05:53:15 -0400 Subject: [Dovecot] Testing process? Message-ID: Morning all! Total newbie question. I'm learning Dovecot and will be plugging in a few things and testing it end to end. What test structures or tools are available for making sure the install is correct and the parameters are taken? Thanks! Leam -- Mind on a Mission From robert at schetterer.org Sat Sep 1 13:30:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 01 Sep 2012 12:30:18 +0200 Subject: [Dovecot] Testing process? In-Reply-To: References: Message-ID: <5041E3BA.3090100@schetterer.org> Am 01.09.2012 11:53, schrieb leam hall: > Morning all! Total newbie question. I'm learning Dovecot and will be > plugging in a few things and testing it end to end. What test structures or > tools are available for making sure the install is correct and the > parameters are taken? > > Thanks! > > Leam > > describe what you mean "install is correct" as there are unnumbered ways to setup dovecot usally you found your errors in the logs, for testing use verbose logging so first plan "what setup" you want to goal, reading dovecot sites , faqs, examples, archives anyway perhaps use some virt. machine for testing multiple stuff -- Best Regards MfG Robert Schetterer From kprprl at gmail.com Sat Sep 1 15:25:19 2012 From: kprprl at gmail.com (PARTH MONGA) Date: Sat, 1 Sep 2012 17:55:19 +0530 Subject: [Dovecot] Testing process? In-Reply-To: <5041E3BA.3090100@schetterer.org> References: <5041E3BA.3090100@schetterer.org> Message-ID: netstat -tulpn port listening on 110(pop) 143(imap) confirms the same and you can also do cat /var/log/dovecot.log rgds p On Sat, Sep 1, 2012 at 4:00 PM, Robert Schetterer wrote: > Am 01.09.2012 11:53, schrieb leam hall: > > Morning all! Total newbie question. I'm learning Dovecot and will be > > plugging in a few things and testing it end to end. What test structures > or > > tools are available for making sure the install is correct and the > > parameters are taken? > > > > Thanks! > > > > Leam > > > > > > describe what you mean "install is correct" > as there are unnumbered ways to setup dovecot > > usally you found your errors in the logs, for testing use verbose logging > > so first plan "what setup" you want to goal, reading dovecot sites , > faqs, examples, archives > > anyway perhaps use some virt. machine for testing multiple stuff > -- > Best Regards > MfG Robert Schetterer > From leamhall at gmail.com Sat Sep 1 17:19:07 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 10:19:07 -0400 Subject: [Dovecot] Testing process? In-Reply-To: References: <5041E3BA.3090100@schetterer.org> Message-ID: > > describe what you mean "install is correct" > > as there are unnumbered ways to setup dovecot > > > > usally you found your errors in the logs, for testing use verbose logging > > > > so first plan "what setup" you want to goal, reading dovecot sites , > > faqs, examples, archives > > > > anyway perhaps use some virt. machine for testing multiple stuff > > -- > > Best Regards > > MfG Robert Schetterer > > > Robert, et al, thank you for reminding me that I need to share what I'm thinking! I've been bouncing it around in my own head but didn't really write enough to communicate. There are a lot of things i need to learn about Dovecot, so first is how to send something to it and then see where it's logged or what actions are taken. My work uses Dovecot and I need to be able to use, test, and understand it. So maybe the first question is, is there a test framework for Dovecot itself or is it better to set up with postfix or something and just route mail through it? Thanks! Leam -- Mind on a Mission From david at wordshell.net Sat Sep 1 21:08:53 2012 From: david at wordshell.net (David Anderson) Date: Sat, 01 Sep 2012 21:08:53 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <1346490929.10359.16.camel@tardis> References: <5041C2FD.1040108@wordshell.net> <1346490929.10359.16.camel@tardis> Message-ID: <50424F35.2040105@wordshell.net> Hi, Thanks. I used monit, and will code something up of that kind - but I was curious as to whether dovecot had something built in; external monitoring is a somewhat blunter tool, as you don't get to know exactly why there was a problem unless you also parse the logs. Tricky to investigate this kind of issue - it has only happened once, and I had a few hundred people trying to log in, so no time to start debugging! But if it recurs I'll be back to ask for guidance... Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net On 01/09/12 12:15, Noel Butler wrote: > Seen similar, we use mon? for network monitoring, our mon pop3 script > is modified to not only connect, but login, if it cant the trigger > becomes active and if fails again at next check, it alerts engineers > by sms. > > Though, I would investigate the core issue, Timo "apparently" fixed > that hung session stuff in 2.0, it was common in 0.x and 1.x series. > > Cheers > > 1: ( > http://sourceforge.net/projects/mon/files/mon/ ) > > > From ml at smtp.fakessh.eu Sun Sep 2 04:59:42 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 02 Sep 2012 03:59:42 +0200 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler Message-ID: <1346551182.14379.1.camel@localhost> Hi Timo in the present work would be possible to walk to another folder for deleted_to_trash please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From me at junc.org Sun Sep 2 08:30:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 07:30:40 +0200 Subject: [Dovecot] =?utf-8?q?How_Converting/Using_courier-userdb_Files_=3F?= In-Reply-To: <5040C15D.4000303@eurodata.de> References: <5040C15D.4000303@eurodata.de> Message-ID: <421fa10c33e339153cb5532da61e57b2@junc.org> Den 2012-08-31 15:51, Dirk Caspari skrev: > how can i convert/use the Original userdb-Files from courier > Installation ? > The files are looking like a "Special"-DB-Format and are not > readable. it can be added if gdbm is supported in dovecot auth backedn, even convert the gdbm db to sqlite will be suported in shells, when its converted lets say to sqlite then its a matter of define how dovecot use it its a one time run From tss at iki.fi Sun Sep 2 17:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 17:28:28 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120831084233.GE3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: On 31.8.2012, at 11.42, Jost Krieger wrote: > One of our users likes to delete attachments from his mails. He is using > roundcube and we installed a plugin for that. Does the plugin change the mbox file directly? > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error. From eddy.ilg at univation.de Sun Sep 2 18:28:26 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:28:26 +0200 Subject: [Dovecot] Special Folder Mocking Message-ID: <50437B1A.8000808@univation.de> Dear Dovecot List, probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. A simple question: Isn't it possible to advertise one imap folder with different names, depending on the client? E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? Best regards, Eddy Ilg From tss at iki.fi Sun Sep 2 18:32:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:08 +0300 Subject: [Dovecot] Mailbox aliases Message-ID: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: .. mailbox Sent { auto = create special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent alias_for = Sent } .. If "Sent Messages" doesn't exist: - DELETE Sent works as usual - RENAME Sent works as usual If "Sent Messages" is created: - it's symlinked to Sent - DELETE "Sent messages" deletes the symlink, not the contents - RENAME "Sent messages" isn't allowed - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. Thoughts? The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Sun Sep 2 18:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:37 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> On 2.9.2012, at 18.28, Eddy Ilg|univation wrote: > Dear Dovecot List, > > > probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. I was just in the middle of writing the previous mail about this :) > A simple question: > Isn't it possible to advertise one imap folder with different names, depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? No. From eddy.ilg at univation.de Sun Sep 2 18:50:12 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:50:12 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> Message-ID: <50438034.4060604@univation.de> > I was just in the middle of writing the previous mail about this :) Already thought you replied and changed the subject of my post ;) >> A simple question: >> Isn't it possible to advertise one imap folder with different names, depending on the client? >> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? > No. But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? A similar thing is "imap_client_workarounds = tb-extra-mailbox-sep". If what I suggested is implementable then what speaks against it? If I have some time I might try writing a patch... Or maybe a dovecot developer can implement it, who better knows the code. Best, Eddy From tss at iki.fi Sun Sep 2 19:39:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 19:39:46 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50438034.4060604@univation.de> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> <50438034.4060604@univation.de> Message-ID: On 2.9.2012, at 18.50, Eddy Ilg|univation wrote: >> I was just in the middle of writing the previous mail about this :) > Already thought you replied and changed the subject of my post ;) >>> A simple question: >>> Isn't it possible to advertise one imap folder with different names, depending on the client? >>> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? >> No. > But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? Even theoretically it's not something I want to consider. It would cause more trouble than gain. (Some clients identify themselves with ID extension, many don't (e.g. Outlook). Some clients might be unreliably detected using the IMAP command tags they use, but that could identify other clients wrong. Different localizations of clients use different mailbox names.) From dovecot at knutejohnson.com Sun Sep 2 20:29:17 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:29:17 -0700 Subject: [Dovecot] Strange messages? Message-ID: <5043976D.7040806@knutejohnson.com> The following messages show up in the output of a script that does some backup. This too is new with the upgrade to Ubuntu server 12.04.1 and dovecot 2.0.19. Anybody know what would generate these sorts of messages? Where or what would cause them? WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Thanks, -- Knute Johnson From dovecot at knutejohnson.com Sun Sep 2 20:30:18 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:30:18 -0700 Subject: [Dovecot] New log entries with 2.0.19? In-Reply-To: <5041436E.7000208@knutejohnson.com> References: <5041436E.7000208@knutejohnson.com> Message-ID: <504397AA.7040300@knutejohnson.com> A couple of days later and it is back to normal. Never mind. knute... On 8/31/2012 4:06 PM, Knute Johnson wrote: > I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded > Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) > disconnect in IDLE and inactivity time outs. I'm using only IMAP with > two users. Both users access the server from multiple devices, phone, > tablet and desktop computer. I had previously set the > max_userip_connections to 20 and rarely got the following limits > exceeded message. That's probably being triggered when we are both home > and using all of our devices on the same network. I've now changed it > to 40 and we'll see how that does. > > The other is the IDLE and inactivity timeouts. I don't really > understand how IDLE works and I couldn't find an inactivity timeout > setting to play with. There are at this second 32 imap or imap-login > processes running with only two phones and one desktop running. That's > a few more than before the upgrade but not a lot. > > --------------------- Dovecot Begin ------------------------ > > Dovecot limits exceeded: > max_userip_connections: penny from 216.240.58.138 to > 216.240.58.140: 56 Time(s) > > Dovecot disconnects: > Disconnected in IDLE: 485 Time(s) > Inactivity: 343 Time(s) > no auth attempts: 14 Time(s) > no reason: 21 Time(s) > > Everything seems to be working OK so maybe I should just not worry about > it? > > Thanks, > > knute... > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_anonymous_username = anonymous > auth_cache_negative_ttl = 1 hours > auth_cache_size = 0 > auth_cache_ttl = 1 hours > auth_debug = no > auth_debug_passwords = no > auth_default_realm = > auth_failure_delay = 2 secs > auth_first_valid_uid = 500 > auth_gssapi_hostname = > auth_krb5_keytab = > auth_last_valid_uid = 0 > auth_master_user_separator = > auth_mechanisms = plain login > auth_realms = > auth_socket_path = auth-userdb > auth_ssl_require_client_cert = no > auth_ssl_username_from_cert = no > auth_use_winbind = no > auth_username_chars = > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ > auth_username_format = > auth_username_translation = > auth_verbose = no > auth_verbose_passwords = no > auth_winbind_helper_path = /usr/bin/ntlm_auth > auth_worker_max_count = 30 > base_dir = /var/run/dovecot > config_cache_size = 1 M > debug_log_path = > default_client_limit = 1000 > default_idle_kill = 60 > default_internal_user = dovecot > default_login_user = dovenull > default_process_limit = 100 > default_vsz_limit = 256 M > deliver_log_format = msgid=%m: %$ > dict_db_config = > director_doveadm_port = 0 > director_mail_servers = > director_servers = > director_user_expire = 15 mins > disable_plaintext_auth = yes > dotlock_use_excl = yes > doveadm_allowed_commands = > doveadm_password = > doveadm_proxy_port = 0 > doveadm_socket_path = doveadm-server > doveadm_worker_count = 0 > first_valid_gid = 1 > first_valid_uid = 500 > hostname = > imap_capability = > imap_client_workarounds = > imap_id_log = > imap_id_send = > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > import_environment = TZ > info_log_path = > instance_name = dovecot > last_valid_gid = 0 > last_valid_uid = 0 > lda_mailbox_autocreate = no > lda_mailbox_autosubscribe = no > lda_original_recipient_header = > libexec_dir = /usr/lib/dovecot > listen = *, :: > lmtp_proxy = no > lmtp_save_to_detail_mailbox = no > lock_method = fcntl > log_path = syslog > log_timestamp = "%b %d %H:%M:%S " > login_access_sockets = > login_greeting = Dovecot ready. > login_log_format = %$: %s > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > login_trusted_networks = > mail_access_groups = > mail_attachment_dir = > mail_attachment_fs = sis posix > mail_attachment_hash = %{sha1} > mail_attachment_min_size = 128 k > mail_cache_fields = flags > mail_cache_min_mail_count = 0 > mail_chroot = > mail_debug = no > mail_fsync = optimized > mail_full_filesystem_access = no > mail_gid = > mail_home = > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_log_prefix = "%s(%u): " > mail_max_keyword_length = 50 > mail_max_lock_timeout = 0 > mail_max_userip_connections = 10 > mail_never_cache_fields = imap.envelope > mail_nfs_index = no > mail_nfs_storage = no > mail_plugin_dir = /usr/lib/dovecot/modules > mail_plugins = > mail_privileged_group = > mail_save_crlf = no > mail_temp_dir = /tmp > mail_uid = > mailbox_idle_check_interval = 30 secs > mailbox_list_index_disable = no > maildir_copy_with_hardlinks = yes > maildir_stat_dirs = no > maildir_very_dirty_syncs = no > master_user_separator = > mbox_dirty_syncs = yes > mbox_dotlock_change_timeout = 2 mins > mbox_lazy_writes = yes > mbox_lock_timeout = 5 mins > mbox_min_index_size = 0 > mbox_read_locks = fcntl > mbox_very_dirty_syncs = no > mbox_write_locks = dotlock fcntl > mdbox_preallocate_space = no > mdbox_rotate_interval = 0 > mdbox_rotate_size = 2 M > mmap_disable = no > passdb { > args = > deny = no > driver = pam > master = no > pass = no > } > pop3_client_workarounds = > pop3_enable_last = no > pop3_fast_size_lookups = no > pop3_lock_session = no > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_save_uidl = no > pop3_uidl_format = %08Xu%08Xv > postmaster_address = > protocols = " imap" > quota_full_tempfail = no > recipient_delimiter = + > rejection_reason = Your message to <%t> was automatically rejected:%n%r > rejection_subject = Rejected: %s > sendmail_path = /usr/sbin/sendmail > service anvil { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = anvil > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 1 > protocol = > service_count = 0 > type = anvil > unix_listener anvil-auth-penalty { > group = > mode = 0600 > user = > } > unix_listener anvil { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service auth-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = auth -w > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener auth-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service auth { > chroot = > client_limit = 4096 > drop_priv_before_exec = no > executable = auth > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener auth-client { > group = > mode = 0600 > user = > } > unix_listener auth-login { > group = > mode = 0600 > user = $default_internal_user > } > unix_listener auth-master { > group = > mode = 0600 > user = > } > unix_listener auth-userdb { > group = > mode = 0600 > user = > } > unix_listener login/login { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service config { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = config > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = config > unix_listener config { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service dict { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dict > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dict { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service director { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = director > extra_groups = > fifo_listener login/proxy-notify { > group = > mode = 00 > user = > } > group = > idle_kill = 4294967295 secs > inet_listener { > address = > port = 0 > ssl = no > } > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener director-admin { > group = > mode = 0600 > user = > } > unix_listener director-userdb { > group = > mode = 0600 > user = > } > unix_listener login/director { > group = > mode = 00 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service dns_client { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dns-client > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dns-client { > group = > mode = 0666 > user = > } > unix_listener login/dns-client { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service doveadm { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = doveadm-server > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener doveadm-server { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service imap-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = imap-login > extra_groups = > group = > idle_kill = 0 > inet_listener imap { > address = > port = 143 > ssl = no > } > inet_listener imaps { > address = > port = 993 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service imap { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = > unix_listener login/imap { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ipc { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = ipc > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener ipc { > group = > mode = 0600 > user = > } > unix_listener login/ipc-proxy { > group = > mode = 0600 > user = $default_login_user > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service lmtp { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = lmtp > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = lmtp > service_count = 0 > type = > unix_listener lmtp { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service log { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = log > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = log > user = > vsz_limit = 18446744073709551615 B > } > service pop3-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = pop3-login > extra_groups = > group = > idle_kill = 0 > inet_listener pop3 { > address = > port = 0 > ssl = no > } > inet_listener pop3s { > address = > port = 0 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service pop3 { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = pop3 > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = > unix_listener login/pop3 { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ssl-params { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = ssl-params > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = startup > unix_listener login/ssl-params { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > shutdown_clients = yes > ssl = yes > ssl_ca = > ssl_cert = ssl_cert_username_field = commonName > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > ssl_client_cert = > ssl_client_key = > ssl_key = ssl_key_password = > ssl_parameters_regenerate = 168 > ssl_verify_client_cert = no > submission_host = > syslog_facility = mail > userdb { > args = > driver = passwd > } > valid_chroot_dirs = > verbose_proctitle = no > verbose_ssl = no > version_ignore = no > protocol imap { > imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID > ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND > UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 ESEARCH > ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS > mail_max_userip_connections = 40 > } > -- Knute Johnson From p at state-of-mind.de Sun Sep 2 23:32:24 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sun, 2 Sep 2012 22:32:24 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <20120902203221.GB18740@state-of-mind.de> Eddy, * Eddy Ilg|univation : > Dear Dovecot List, > > > probably everyone knows the problem that different clients use > different names for "Sent", "Trash" and so on. > > A simple question: > Isn't it possible to advertise one imap folder with different names, > depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent > Elements" (translated from german) and if the client is Thunderbird, > show the "Sent" folder as "Sent"? the answer is the IMAP extension "SPECIAL USE". We sponsored SPECIAL USE in Dovecot and in Thunderbird. Timo implemented it in Dovecot autumn 2011 and Ben Bucksch added SPECIAL USE support in Mozilla shortly after. Sadly Mozilla has decided to turn Thunderbird down - i.e. hand it over to some 'community' - and I am not sure our contribution it will be released as Mozilla claims they will not add any new features anymore. IF they would you could tell Dovecot to name the SPECIAL USE folders like Outlook expects them and Thunderbird to map its folder on top. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Mon Sep 3 00:01:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 02 Sep 2012 23:01:16 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <5043C91C.2070608@schetterer.org> Am 02.09.2012 17:32, schrieb Timo Sirainen: > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. Hi Timo, funny, yesterday i had problems with this, i am using virtual plugin setup with pop3 also disallow i.e imap Sent/Trash/Drafts download via imap, but allow Junk imap folder to be downloaded via pop3 cause i have a global sieve rule storing, that spam tagged mail is going direct in Junk folder, that usally cant be overided by users. This should avoid redirecting spam tagged mail via sieve by users in prime. This works nice. My users use all kind of os and mail clients typical ISP Setup Default webmail Horde Imp layout is configured for thunderbirds default imap folder layout, also thunderbird is announced to be the supported mail client via the faq help site, not because its the best client appearing, only for ,that it is installable in all major os, so debug is possible widly In the help site and pictured welcome pdf mail there are setups shown for all major mail clients, as outlook, thunderbird , winmail. Not for Apple cause i havent got the chance to make screenshots from it yet. I also have active sync via z-push so setup in android mail app is on the help site too. So far so good. With Problems that i.e outlook is very different in handling imap folder setup through its versions As more and more people going to use mobile, stuff get more complicated As one user uses k9mail android vers 4.11 ( this is recent ), which cant do imap special use yet, so a i.e german Sent Folder ( Gesendet ) is created auto by the k9 client, same user does pop3 via outlook so the problem appeared that he send mail with copy in the "Gesendet" from K9 mail android folder ,but this Folder wasnt disallowed in the pop3 virtual layout , so a loop appeared. I had now to disallow folder "Gesendet/Papierkorb/Entw?rfe" for pop3. Upcomming Versions K9 Mail 4.2 can do imap special use, and it works by testing Only small real World example. I investigated little, and found neither Thunderbird Outlook or Apple is supporting imap special use yet ( seems its on the road for thunderbird ), perhaps someone knows more about this. My meaning to this theme is, its simply teribble for 21 century mail that there is no recommended default imap folder layout for all mailclients, imap special use looks promising , hopefully it will wide spread Goal: Configure mail should be done auto only with emailadress and password including default imap folder layout and match trough all major mail imap clients. So example wiki site for configure dovecot relate to major imap clients and their versions behave in default imap folder layout and support of imapspecial use would be extrem helpfull, in fact yesterday i searched about that. -- Mfg Best Regards Robert Schetterer From me at junc.org Mon Sep 3 00:22:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 23:22:40 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> Message-ID: <0fa581317d82ab7f071329317125bc55@junc.org> Den 2012-08-24 07:18, Voytek Eymont skrev: > I want to offer 'plain' unsecure pop/imap access, just for temp. > testing purpose, do I simply comment out > ssl = required plain is not related to ssl enabled, here i have disabled pop3 imap, so now here i only supports ssl / tls connections better set listen to localhost or another rfc1918 ip From h.reindl at thelounge.net Mon Sep 3 00:32:34 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 02 Sep 2012 23:32:34 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <5043C91C.2070608@schetterer.org> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> <5043C91C.2070608@schetterer.org> Message-ID: <5043D072.4020604@thelounge.net> Am 02.09.2012 23:01, schrieb Robert Schetterer: > Goal: > Configure mail should be done auto only with emailadress and password > including default imap folder layout and match trough all major mail > imap clients. > > So example wiki site for configure dovecot relate to major imap clients > and their versions behave in default imap folder layout and support of > imapspecial use would be extrem helpfull, in fact yesterday i searched > about that. the real problem are stupid clients at all the folders just could called "Sent" on the server and the client can display whatever it wants, this even works if you take some minutes to configure all your clients to use the same folder and the apple is coming: eahc different version of their clients is creating different sent folders - after update MacOSX it happens sometimes that it creates a new incarnation on the servr and spit new messages in the new one instead the over months used existing and if a users owns different apple-devices he ends up in many incarnations * Sent * Sent Messages * Sent Messages (acount name) * Gesendet and this is only the example of ONE apple-only-user you can imagine what happens using different other clients and smartphone-types this is nothing that can be fixed on the server side the idiotic client developers should take care of the different possible existing special folders and use any of them if existing instead create a new one -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 01:31:11 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:31:11 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0fa581317d82ab7f071329317125bc55@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> Message-ID: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From raubvogel at gmail.com Mon Sep 3 01:38:53 2012 From: raubvogel at gmail.com (Mauricio) Date: Sun, 2 Sep 2012 22:38:53 +0000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> Message-ID: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> I thought you could specify ssl to be wanted but not required, but I might be thinking on client side. Sent via iPhone -----Original Message----- From: "Voytek Eymont" Sender: dovecot-bounces at dovecot.orgDate: Mon, 3 Sep 2012 08:31:11 To: Subject: Re: [Dovecot] Disable security for testing? On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From voytek at sbt.net.au Mon Sep 3 01:44:16 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:44:16 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <710bc96f1513f58dd51935702626803e.squirrel@sbt.net.au> On Mon, September 3, 2012 8:38 am, Mauricio wrote: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. Sent via iPhone I'm happy to have it as a default requirement, seems like a reasonable requirement, this was just for some temp testing/verification etc -- Voytek From me at junc.org Mon Sep 3 02:26:20 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:26:20 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <0535537af33849715f4b3efd97569d4d@junc.org> Den 2012-09-03 00:38, Mauricio skrev: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. the point is ? > Sent via iPhone i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port 110/143 anymore From h.reindl at thelounge.net Mon Sep 3 02:32:57 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 01:32:57 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0535537af33849715f4b3efd97569d4d@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> Message-ID: <5043ECA9.5010005@thelounge.net> Am 03.09.2012 01:26, schrieb Benny Pedersen: > i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port > 110/143 anymore why? STARTTLS is 110/143 per definition! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From me at junc.org Mon Sep 3 02:40:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:40:56 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <5043ECA9.5010005@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: Den 2012-09-03 01:32, Reindl Harald skrev: > STARTTLS is 110/143 per definition! EOD From adi at ddns.com.au Mon Sep 3 02:41:14 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 09:41:14 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> Message-ID: <20120903094114.2a75835d@adi.ddns.local> On Fri, 31 Aug 2012 17:11:07 +0300 Timo Sirainen wrote: > > The POP/IMAP part is working fine. What I'm trying to do is to use > > Dovecot SASL implementation in Postfix to do SMTP authentication in > > a similar manner. The problem I have with my current configuration > > is that SMTP authentication succeeds if only the username matches, > > because password forwarding works if the authentication succeeds > > with any given password, as documented at > > http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy > > Dovecot has no SMTP proxy (currently). And anyway Postfix doesn't use > SMTP to do authentication, Postfix authenticates using Dovecot's > internal protocol, which replies that Postfix should do the proxying, > which it of course doesn't do. Yes, I know and that's exactly what I was trying to do: use Dovecot authentication method in Postfix to authenticate the user. Because of the fact that Dovecot doesn't do SMTP authentication, I was thinking of a way of using its authentication service by getting the SMTP login credentials from the backend POP/IMAP server. The request may sound a bit unusual, but in our case it makes sense. For our setup we currently run: - a farm of backend SMTP/POP/IMAP servers that are hosting the mailboxes and where the user credentials are managed. They are running Courier IMAP. - a group of SMTP/POP/IMAP proxies. These proxies are currently replicating the login credentials from the backend servers and the routing to the backends using a local database. Perdition is currently the POP/IMAP proxy, but having it replaced with Dovecot would help us in getting the password forwarding to the backends running, which means we wouldn't need to store the credentials on the proxy, only the user->host routing entries. Are there any plans to have Dovecot authentication service to do SMTP authentication against IMAP or POP3 proxy provided information? This, of course, means we'd have the authentication result tied to the response of the backend IMAP/pop3 server. > > My question is, given the above: is there a way to get SMTP > > authentication properly in this scenario? > > Make Postfix authenticate against the backend Dovecot server. You'll > need to setup service auth { inet_listener } to some port for it. The POP/IMAP backends are running Courier IMAP, as I've just mentioned and due to the existing hosting environment it's very unlikely to replace it with something else. -- Adi Pircalabu, System Administrator Discount Domain Name Services Pty Ltd, a Total Internet Company PO Box 887, Hawthorn Vic 3122, Australia, T +61 3 9815 6868 Ask me about cloud hosting services From tss at iki.fi Mon Sep 3 02:47:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 02:47:48 +0300 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <20120903094114.2a75835d@adi.ddns.local> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: On 3.9.2012, at 2.41, Adi Pircalabu wrote: > Are there any plans to have Dovecot authentication service to do SMTP > authentication against IMAP or POP3 proxy provided information? This, > of course, means we'd have the authentication result tied to the > response of the backend IMAP/pop3 server. I don't see any clean way of doing that. >>> My question is, given the above: is there a way to get SMTP >>> authentication properly in this scenario? >> >> Make Postfix authenticate against the backend Dovecot server. You'll >> need to setup service auth { inet_listener } to some port for it. > > The POP/IMAP backends are running Courier IMAP, as I've just mentioned > and due to the existing hosting environment it's very unlikely to > replace it with something else. Maybe use IMAP authentication as the backend? pam_imap at least can do that. Or you can already also use Dovecot v2.1's passdb imap to do this, pretty much equivalent to pam_imap. From ml at smtp.fakessh.eu Mon Sep 3 02:55:20 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 01:55:20 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: <32bbc94eb73b728d0a4646c6d7a3b67d@smtp.fakessh.eu> I currently work this plugin with several folders nice a documentation include Le 2012-09-02 03:59, ml a ?crit?: > Hi Timo > > in the present work would be possible to walk to another folder for > deleted_to_trash > > please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From me at junc.org Mon Sep 3 02:55:21 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:55:21 +0200 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Den 2012-09-03 01:47, Timo Sirainen skrev: > Maybe use IMAP authentication as the backend? pam_imap at least can > do that. Or you can already also use Dovecot v2.1's passdb imap to do > this, pretty much equivalent to pam_imap. drop pam, and use saslauthd with remote imap, or setup cyrus sasl with sql/ldap/whatever one needs, just in case one more ask why i did not use dovecot :) From tss at iki.fi Mon Sep 3 03:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 03:18:22 +0300 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: On 2.9.2012, at 4.59, ml wrote: > in the present work would be possible to walk to another folder for > deleted_to_trash I suppose, with some code modifications. But I'd prefer if nobody used this plugin (or anything equivalent to its functionality). From ml at smtp.fakessh.eu Mon Sep 3 03:26:54 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 02:26:54 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: References: <1346551182.14379.1.camel@localhost> Message-ID: <303d5f8de294c07f547603b58a4aec85@smtp.fakessh.eu> Le 2012-09-03 02:18, Timo Sirainen a ?crit?: > On 2.9.2012, at 4.59, ml wrote: > >> in the present work would be possible to walk to another folder for >> deleted_to_trash > > I suppose, with some code modifications. But I'd prefer if nobody > used this plugin (or anything equivalent to its functionality). until the it works very well as it is with an inclusion in dovecot.conf. and why not use it I have folders in my email that I am happy with this empty nice plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From adi at ddns.com.au Mon Sep 3 04:55:20 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 11:55:20 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Message-ID: <20120903115520.218c8190@adi.ddns.local> On Mon, 03 Sep 2012 01:55:21 +0200 Benny Pedersen wrote: > Den 2012-09-03 01:47, Timo Sirainen skrev: > > > Maybe use IMAP authentication as the backend? pam_imap at least can > > do that. Or you can already also use Dovecot v2.1's passdb imap to > > do this, pretty much equivalent to pam_imap. I had a look at pam_imap and had to fix the spec file to get it built for CentOS 6 64bit. For the record, I had to add the following in the %build section: export CFLAGS="%{optflags} -fPIC" export CXXFLAGS="%{optflags} -fPIC" However: > drop pam, and use saslauthd with remote imap, or setup cyrus sasl > with sql/ldap/whatever one needs Yep, saslauthd with rimap appears to be exactly what I need. > just in case one more ask why i did not use dovecot :) The password forwarding feature is one of the reasons I started looking into Dovecot. Thanks to you both for your help. -- Adi Pircalabu, System Administrator From amateo at um.es Mon Sep 3 09:48:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 08:48:54 +0200 Subject: [Dovecot] exclude a folder for lazy expunge (move to trash) In-Reply-To: <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> References: <503491C4.7030208@um.es> <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> Message-ID: <504452D6.8@um.es> El 31/08/12 15:23, Timo Sirainen escribi?: > On 22.8.2012, at 11.01, Angel L. Mateo wrote: > >> I'm configuring lazy expunge plugin to act as a kind of backup of my user's emails. >> >> I'm getting a little problem because a lot of my users use "move to trash" behaviour in their email client (most of them are using thunderbird which has no option to hide deleted messages and is a hassle working with mark as deleted with it). The problem combining move to trash with lazy expunge is that messages are stored in the expunge namespace twice, first when it was deleted from the original folder and the second when is deleted from the trash. >> >> So... is there any way to exclude trash folder from lazy expunge plugin, so mails are stored in the expunge namespace only when they are deleted from their original folder and not from the trash? > > Not right now. Maybe it could be automatically done when SPECIAL-USE is enabled and the deletion is from a \Trash mailbox. > >> Another idea... We are purging trash folders with a cron running "doveadm purge" commands. Is there any way to run this command disabling lazy expunge plugin? > > doveadm purge shouldn't affect lazy expunge, but you can disable lazy_expunge by using something like doveadm -o mail_plugins=everything,except,lazy_expunge expunge .. > Sorry, I meant "doveadm expunge" (I always confuse these two commands) which does affect lazy expunge -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From alec at alec.pl Mon Sep 3 10:02:13 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 03 Sep 2012 09:02:13 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <504455F5.7000504@alec.pl> On 09/02/2012 05:32 PM, Timo Sirainen wrote: > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > Thoughts? Yes. It should solve some problems, but I think that only wide-use of SPECIAL-USE is a solution for this. Also.... I don't use current version of dovecot, but from reading this list I remember that SPECIAL-USE implementation in dovecot is read-only. What I'd like to see is possibility to set special-use flags by the client - no server configuration required. It means, when user connects to his mailbox for the first time and special folders doesn't exist, the client (supporting SPECIAL-USE) can create folders according to its configuration and set their special-use flags. Or at any time when it find that special folder doesn't exist. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From zucca at systemschmiede.com Mon Sep 3 10:01:51 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Mon, 03 Sep 2012 09:01:51 +0200 Subject: [Dovecot] Imap Ghost folder In-Reply-To: <292te8ob8kv8@mids.svenhartge.de> References: <50341D40.9090001@systemschmiede.com> <50353D32.2080401@systemschmiede.com> <5038B2C5.3090408@systemschmiede.com> <4CE233FA-2954-4C39-9F79-13C12ADFDCF2@iki.fi> <5038CB71.5010808@systemschmiede.com> <503C818A.4010203@systemschmiede.com> <503DE2D9.3080300@systemschmiede.com> <20120829204814.GA9265@daniel.localdomain> <503F0B77.5030206@systemschmiede.com> <504072FA.8020606@systemschmiede.com> <56D9DD4B-ED96-47C6-96F4-322D6907508C@iki.fi> <5040C32C.2020706@systemschmiede.com> <85E5DEFA-D38E-441E-BFE9-64FC820069E8@iki.fi> <5040C789.8000502@systemschmiede.com> <292te8ob8kv8@mids.svenhartge.de> Message-ID: <504455DF.4020202@systemschmiede.com> >> hm, but 2.1.9 is not in backports yet, is it? > And will not be for some time. 2.1.9 is not even in Unstable and even if > it were, it would not migrate to testing (and be an eligible backporting > target) due to the freeze before the Wheezy release. So, no workaround for this? Noone can reproduce it using my configuration? Very strange.. Thanks Sascha. From amateo at um.es Mon Sep 3 11:41:24 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 10:41:24 +0200 Subject: [Dovecot] doveadm command to block a user? Message-ID: <50446D34.5000702@um.es> Hello, Is there any command to be able to lock imap access to a specific user? With doveadm kick I could close his connections, but I want to avoid future ones too and just for imap. Any way to do it? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tseveendorj at gmail.com Mon Sep 3 12:10:14 2012 From: tseveendorj at gmail.com (tseveendorj) Date: Mon, 03 Sep 2012 17:10:14 +0800 Subject: [Dovecot] dovecot sasl + postfix issue Message-ID: <504473F6.5010205@gmail.com> Hello, I'm trying to configure postfix + Dovecot SASL for user authenticated mail relay. I set following configuration on postfix queue_directory = /var/spool/postfix smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_authenticated_header = yes smtpd_sasl_security_options = noanonymous smtpd_sasl_local_domain = $myhostname broken_sasl_auth_clients = yes smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination Following are dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_mechanisms = plain login digest-md5 base_dir = /var/run/dovecot/ mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r when I see telnet localhost 25 eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 beastie ESMTP Postfix (Ubuntu) ehlo beastie 250-beastie 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN quit 221 2.0.0 Bye Connection closed by foreign host. there is no 250-AUTH How do I fix this ? From Bill at knoxvillechristian.org Mon Sep 3 12:36:32 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Mon, 03 Sep 2012 05:36:32 -0400 Subject: [Dovecot] dovecot sasl + postfix issue In-Reply-To: <504473F6.5010205@gmail.com> References: <504473F6.5010205@gmail.com> Message-ID: <50447A20.7060900@knoxvillechristian.org> You should get AUTH after you STARTTLS. Bill On 9/3/2012 5:10 AM, tseveendorj wrote: > Hello, > > I'm trying to configure postfix + Dovecot SASL for user authenticated > mail relay. > > I set following configuration on postfix > > queue_directory = /var/spool/postfix > smtpd_sasl_auth_enable = yes > smtpd_sasl_type = dovecot > smtpd_sasl_path = private/auth > smtpd_sasl_authenticated_header = yes > smtpd_sasl_security_options = noanonymous > smtpd_sasl_local_domain = $myhostname > broken_sasl_auth_clients = yes > smtpd_recipient_restrictions = reject_unknown_sender_domain, > reject_unknown_recipient_domain, reject_unauth_pipelining, > permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination > > Following are dovecot -n > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_mechanisms = plain login digest-md5 > base_dir = /var/run/dovecot/ > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_cipher_list = > ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to <%t> was automatically rejected:%n%r > > > > when I see telnet localhost 25 > > eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > 220 beastie ESMTP Postfix (Ubuntu) > ehlo beastie > 250-beastie > 250-PIPELINING > 250-SIZE 10240000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > quit > 221 2.0.0 Bye > Connection closed by foreign host. > > there is no 250-AUTH > > How do I fix this ? From h.reindl at thelounge.net Mon Sep 3 13:05:10 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:05:10 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: <504480D6.1000900@thelounge.net> Am 03.09.2012 01:40, schrieb Benny Pedersen: > Den 2012-09-03 01:32, Reindl Harald skrev: > >> STARTTLS is 110/143 per definition! > > EOD oh yeah this is the right attitude * open thunderbird * go to server settings * switch between STARTTLS/SSL * look how the default port changes for SMTP port 465 is even deprecated and you should use 587 (submission) with STATTLS for endusers but do what you want if you like support-calls as example from customers sitting in a network where most outgoing ports are blocked (including 993 and 995) while 143 is open, yes i have seen this from customers sitting in gonvernment networks but hey, for you this is EOD because you know it better SMTP STARTTLS: 587 SMTP over SSL: 465 POP3 STARTTLS: 110 POP3 over SSL: 995 IMAP STARTTLS: 143 IMAP over SSL: 993 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 13:15:27 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 20:15:27 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <504480D6.1000900@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> Message-ID: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> > but hey, for you this is EOD because you know it better > > SMTP STARTTLS: 587 > SMTP over SSL: 465 > POP3 STARTTLS: 110 > POP3 over SSL: 995 > IMAP STARTTLS: 143 > IMAP over SSL: 993 in my limited experience, when I tried to set Outlook Express clients with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got it on 993) question: is it because Outlook Express does SSL only, but not TLS? (or is it simply because it's crappy application ?) thanks, enquiring minds need to know, -- Voytek From h.reindl at thelounge.net Mon Sep 3 13:19:16 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:19:16 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <50448424.7000804@thelounge.net> Am 03.09.2012 12:15, schrieb Voytek Eymont: >> but hey, for you this is EOD because you know it better >> >> SMTP STARTTLS: 587 >> SMTP over SSL: 465 >> POP3 STARTTLS: 110 >> POP3 over SSL: 995 >> IMAP STARTTLS: 143 >> IMAP over SSL: 993 > > > in my limited experience, when I tried to set Outlook Express clients with > my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; > (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got > it on 993) it's a microsoft problem > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) because it is a crappy application you see htunderbird perfectly switching between STARTTLS/SSL ports you can configure each webmail i know to use STARTTLS you can configure even Apple Mail for both usecases -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From amateo at um.es Mon Sep 3 14:16:35 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 13:16:35 +0200 Subject: [Dovecot] Syntax for doveadm auth cache Message-ID: <50449193.8080101@um.es> Hello, What is the correct syntax of doveadm auth cache command? According to command line help: usage: doveadm [-Dv] [-f ] [] altmove [-u |-A] [-S ] [-r] auth [-a ] [-x ] [] auth cache but when I try to run doveadm auth cache, then I'm asked for a password, so I think the command is trying to authenticate cache user. Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): doveadm auth cache flush 4 cache entries flushed So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From sven at svenhartge.de Mon Sep 3 14:31:18 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 3 Sep 2012 13:31:18 +0200 Subject: [Dovecot] finding messages deleted before timespec Message-ID: <4934p48so7v8@mids.svenhartge.de> Hi! I've been pondering about doveadm-search-query(7) and trying to formulate a search query for some time now and it seems there is no way to search for messages, which have been marked DELETED before a certain time. I want to use this to expunge all DELETED messages which have been in that state for 24h or longer. For example a cronjob runs at midnight to expunge those messages but a user deletes one just 10 minutes before that, this message should not be deleted until the day after. This is my first time diving deeper into IMAP search queries so I have the nagging feeling I am missing something important here. Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel at dhag.net Mon Sep 3 16:32:55 2012 From: daniel at dhag.net (Daniel Funke) Date: Mon, 03 Sep 2012 15:32:55 +0200 Subject: [Dovecot] dovecot proxy ssl-parameter from database? Message-ID: <5044B187.6000609@dhag.net> Hi List, I want to migrate my IMAP-users from an old Courier-IMAP-Server to a new dovecot-Server. The old server supports IMAP without and with TLS. The new one only with TLS. To users should be migrated one by one with changing some client parameters. To leave the servername of the imap server I want to use the dovecot proxy during migration. the proxy uses a sql db to look for the old or new host for every user. So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) I tried with http://wiki2.dovecot.org/UserDatabase/ExtraFields and returned a field "userdb_ssl" ("required" for host dovecot, "yes" for host courier). But it doesn't work. A migrated user can still login without tls to the proxy and the dovecot backend. Any ideas? Daniel Here is my doveconf -n # 2.1.9: /opt/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login default_process_limit = 500 disable_plaintext_auth = no listen = * managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /opt/dovecot/etc/dovecot/sql.conf driver = sql } protocols = imap pop3 service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3s { port = 0 } } ssl_cert = References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <20120903160119.06de7fcd@zetkin.int.primekey.se> On Mon, 3 Sep 2012 20:15:27 +1000 "Voytek Eymont" wrote: > in my limited experience, when I tried to set Outlook Express clients > with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on > 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on > 143, but, got it on 993) > > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) The ports 143 and 993 are used in different way when you want to have TLS. In case of 143, you begin the communication in unencrypted mode, then tell the server that you want to initiate a TLS handshake (you literally send a STARTTLS string to it). After that both client and server start negotiating the TLS connection (and hopefully they manage to do it :). With the port 993 the expected/standard behavior is that you begin the TLS handshake right upon establishing the connection. Normally the servers will not accept any unencrypted IMAP command on it (in other words, you don't send STARTTLS to the server, the server expects you to start the handshake right away). Similar concept is implemented for POP3, SMTP, LDAP, XMPP, and many others... Btw, SSL and TLS are kind of the same things (or to be more precise when people are talking about it they refer to the same thing, unless they're crypto techies which are using the terminology correctly :) Best regards P.S. It's quite probably a crappy application as well :) -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From Jost.Krieger+dovecot at rub.de Mon Sep 3 17:11:21 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 3 Sep 2012 16:11:21 +0200 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: <20120903141121.GR3002@ruhr-uni-bochum.de> [Sorry about the personal reply.] On Sun, Sep 02, 2012 at 05:28:28PM +0300, Timo Sirainen wrote: > > On 31.8.2012, at 11.42, Jost Krieger wrote: > > > One of our users likes to delete attachments from his mails. He is using > > roundcube and we installed a plugin for that. > > Does the plugin change the mbox file directly? No, it just uses IMAP. > > > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) > > This looks like it does. Existing emails must not be changed, IMAP > protocol forbids it (and Dovecot internals don't support it either). > The only safe way to drop attachments is to assign a new UID for the > mail, which basically means saving a new message without attachments > and deleting the old one. The Content-Length header doesn't really > have anything to do with this error. Sorry if I was unclear. This is exactly what the plugin does. It builds a new message, starting with the old message and leaving out the unwanted attchment(s). Then it saves the new message, deletes the old message and finally copies over the flags. I had a look at the handling of "Content-Length" in dovecot and had the impression that a new header was added but the old was not removed (if formally ok) and on reading the first header wins. But I'm not sure, of course. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From kgc at corp.sonic.net Mon Sep 3 21:26:55 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:26:55 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected Message-ID: <5044F66F.3010505@corp.sonic.net> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) I've dropped one of the servers out of the ring to prevent suffering but this is a less than ideal situation. Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 10000 director_mail_servers = 1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 director_servers = a.director.foo b.director.foo disable_plaintext_auth = no doveadm_proxy_port = 1842 login_trusted_networks = 10.0.0.1 mbox_write_locks = fcntl passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 20103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_ca = Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:23:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla2 host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:23:32 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:24:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:05 a.director. a dovecot: director: Warning: Delaying new user requests until ring is synced >Sep 3 09:24:32 a.director. a dovecot: director: Warning: Ring is synced, continuing delayed requests >Sep 3 09:24:41 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:24:41 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:41 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:24:41 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:25:11 b.director. b dovecot: director: Error: User hash 2285697953 is being redirected to two hosts: 10.10.10.39 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689467) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 623192092 is being redirected to two hosts: 10.10.10.76 and 10.10.10.39 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 1683990717 is being redirected to two hosts: 10.10.10.43 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:25:12 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director lef -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From robert at schetterer.org Mon Sep 3 21:32:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 03 Sep 2012 20:32:59 +0200 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <5044F7DB.9080601@schetterer.org> Am 03.09.2012 20:26, schrieb Kelsey Cummings: > I've had 2x director ring up and running with production load on 2.1.8 > with around 10,000 active connections for two weeks and everything has > been working great - until this morning. meanwhile try upgrade to 2.1.9 i cant see big changes there for director which should brake something http://hg.dovecot.org/dovecot-2.1/rev/bc86680293d2 however its better debug with latest patchlevel -- Best Regards MfG Robert Schetterer From eddy.ilg at univation.de Mon Sep 3 21:36:22 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Mon, 03 Sep 2012 20:36:22 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. Message-ID: <5044F8A6.7060200@univation.de> Hi, In the documentation of dovecot-antispam it is mentioned that moving mails out of the spam folders trains them as non-spam. I.e. thunderbird automatically deletes old spam messages (after a time limit, i.e. 14 days). Will this also train the old spam messages as non-spam? Cheers, Eddy From kgc at corp.sonic.net Mon Sep 3 21:37:57 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:37:57 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F7DB.9080601@schetterer.org> References: <5044F66F.3010505@corp.sonic.net> <5044F7DB.9080601@schetterer.org> Message-ID: <5044F905.60204@corp.sonic.net> On 9/3/2012 11:32 AM, Robert Schetterer wrote: > Am 03.09.2012 20:26, schrieb Kelsey Cummings: >> I've had 2x director ring up and running with production load on 2.1.8 >> with around 10,000 active connections for two weeks and everything has >> been working great - until this morning. > > meanwhile try upgrade to 2.1.9 I did that in the second sentence. ;) One of the servers had a single segfault but this wasn't the original cause and didn't show up until after I was poking. director[30709]: segfault at 40 ip 0000000000405d37 sp 00007fff42af80e0 error 6 in director[400000+f000] -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Mon Sep 3 22:06:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 22:06:49 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. > > There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) Did you try stopping both and then starting them again? That clears up all the state they have. > Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. Directors themselves think they're having trouble connecting to each others.. Annoyingly it doesn't give specific error messages about what happened. I should improve the logging.. If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. From kgc at corp.sonic.net Tue Sep 4 04:58:16 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 18:58:16 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <50456038.4070008@corp.sonic.net> On 9/3/2012 12:06 PM, Timo Sirainen wrote: > Did you try stopping both and then starting them again? That clears up all the state they have. I'm not sure that they were both down when restarting them and will try this tonight. > If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. I can't rule that out but there are not any signs that there are any hardware, OS or network related issues. Thanks for gettting the ring status into doveadm by the way. At least our monitoring caught this quickly. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From bmalik at ihlas.net.tr Tue Sep 4 07:15:23 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Mon, 3 Sep 2012 21:15:23 -0700 (PDT) Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <20120814230403.GA11246@daniel.localdomain> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> Message-ID: <1346732123552-37388.post@n4.nabble.com> i am sorry when i login to the server using pop3s, i see in the dovecot log; @4000000050457dd9179c653c pop3-login: Info: Login: user=, method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, session= if i add my mail address into the users-with-forced-encryption file, i would not reach my account. i get an error; @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script denied access to user test at byserver.net whereas i connect to the account via pop3s if i remove my mail address from the users-with-forced-encryption there is no problem. my postlogin_pop3.sh; #!/usr/local/bin/bash if [ "$SECURED" == "secure" ] then :# handle secure connections here else :# handle insecure connections here if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption then printf "* NO [ALERT] You are not allowed to connect without encryption\r\n" exit 0 fi fi exec "$@" ########### there is also no problem related to SSL certificate what can be the issue ? -- View this message in context: http://dovecot.2317879.n4.nabble.com/about-postlogin-in-dovecot2-tp5993p37388.html Sent from the Dovecot mailing list archive at Nabble.com. From bangkokmaco at gmail.com Tue Sep 4 07:21:54 2012 From: bangkokmaco at gmail.com (cc young) Date: Mon, 3 Sep 2012 21:21:54 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting Message-ID: <1346732514245-37389.post@n4.nabble.com> cannot get TB to recognize either pop3/s or imap/s server can connect just fine with: openssl s_client -connect ms1.myserver.net:993 . login ... but trying with TB /var/log/mail.log gets: dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx -- View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html Sent from the Dovecot mailing list archive at Nabble.com. From me at junc.org Tue Sep 4 08:00:35 2012 From: me at junc.org (Benny Pedersen) Date: Tue, 04 Sep 2012 07:00:35 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. In-Reply-To: <5044F8A6.7060200@univation.de> References: <5044F8A6.7060200@univation.de> Message-ID: Den 2012-09-03 20:36, Eddy Ilg|univation skrev: > In the documentation of dovecot-antispam it is mentioned that moving > mails out of the spam folders trains them as non-spam. I.e. > thunderbird automatically deletes old spam messages (after a time > limit, i.e. 14 days). Will this also train the old spam messages as > non-spam? i hope it depends on dovecot-antispam knows the trashcan or not, but seen from here i think this is really a bug, here i use v1 of dovecot and i dont know if this is changed in v2 dovecot if needs be post plugin config, here i am unsure about this aswell :/ From stsiol at yahoo.co.uk Tue Sep 4 08:59:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 06:59:47 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: cc young > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 7:21 > Subject: [Dovecot] thunderbird not connecting > > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: > http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. > Thunderbird is not the best of clients out there. It's ok but not the best. I am using TB on my sites and found out that it has many issues. Can you be a bit more specific ? Are you trying to connect to Secure IMAP ? (993) on dovecot ? In that case, as far as I remember you have to "fool" Thunderbird during the mail account set up process.? It's a bit crazy, I know but it's true. I have documented everything. Let me know where the hickup is and I'll try to get the docs and let you know how to work around this. HTH, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Sep 4 10:31:22 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 08:31:22 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 9:42 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 12:59 PM, Spyros Tsiolis wrote: > > >> >>----- Original Message ----- >>> From: cc young >>> To: dovecot at dovecot.org >>> Cc: >>> Sent: Tuesday, 4 September 2012, 7:21 >>> Subject: [Dovecot] thunderbird not connecting >>> >>> cannot get TB to recognize either pop3/s or imap/s server >>> >>> can connect just fine with: >>> >>> openssl s_client -connect ms1.myserver.net:993 >>> . login ... >>> >>> but trying with TB /var/log/mail.log gets: >>> >>> dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html >>> Sent from the Dovecot mailing list archive at Nabble.com. >>> >> >> >>Thunderbird is not the best of clients out there. >>It's ok but not the best. >> >>I am using TB on my sites and found out that it has many issues. >>Can you be a bit more specific ? >>Are you trying to connect to Secure IMAP ? (993) on dovecot ? >> >>In that case, as far as I remember you have to "fool" Thunderbird >>during the mail account set up process.? >> >>It's a bit crazy, I know but it's true. I have documented everything. >>Let me know where the hickup is and I'll try to get the docs and >>let you know how to work around this. >> >>HTH, >> > > >right - ??trying to connect to Secure IMAP ? (993) on dovecot > > >not in love with thunderbird, but need to access via linux and ms > > >any help / insight would be wonderful > >? Hi again, Taken from my docs . This is the first part for Mozilla Thunderbird setup. I've seen this work both on Win32 systems and linux systems (mainly Linux Mint). There's a part on pop3/smtp. I am sure you can skip that and do your own magic for those two protocols :? Pre-Installation, Wizard Configuration -------------------------------------- During initial Thunderbird startup, the admin is greeted by a welcome window named ?Mail Account Setup?? Click on Cancel Installation ------------ Click on ?Edit? -> ??Account Settings? New window pops up Click on ?Add? New window pops up Fill in ?Description? (e.g. ?Name Surname mailbox?) On ?Server Name? enter the IP address (e.g. 192.168.3.5) On ?Port? enter the SMTP port (e.g. 25) On ?Security and Authentication? ? ?Connection Security? enter none On ?Authentication Method? choose Password transmitted insecurely On ?Username? enter the users' username including the FQDN (e.g. name at domain.gr) Click on OK From ?Edit?, click on ?Account Settings? From ?Account Actions? click on ?Add Mail Account? New windows pops up Enter in the same name for ?Your name? Enter in the same e-mail address for ?Email address? On ?Password? enter in users' password Click on ?Continue? then without waiting click on ?Manual Config? Once ?Manual Config? is clicked, thunderbird will allow the admin to perform additional configuration tasks For ?Incoming? the admin will choose ?IMAP?, Server Hostname will be set to the servers' IP address (again, 192.168.3.5) From ?Port? choose ?993? From ?SSL? choose ?SSL/TLS? From ?Authentication? choose ?Autodetect? For ?Outgoing? the admin will let the ?SMTP? option For ?Server Hostname?, choose the same IP address (e.g. 192.168.3.5) For ?Port? choose ?25? For ?SSL? choose ?None? For ?Authentication? choose ?Normal Password? On the Username field, enter the full user name, e.g. ?user at domain.gr? Click on ?Re-test? Click on ?Create Account? Check on the ?I understand the Risks? checkbox? Click on ?Create Account? Click on ?Confirm Security Exception? Click on ?OK? Hope this helps and best regards, Spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From bangkokmaco at gmail.com Tue Sep 4 10:50:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Tue, 4 Sep 2012 14:50:40 +0700 Subject: [Dovecot] problems with ssl cert Message-ID: in conf.d/10-ssl.conf: ssl = yes ssl_cert = I upgraded dovecot from 1.1 to 2.1.8 pop3/imap works but i have a problem. i use pop3 protocol on outlook 2010 for my account. and i set keep my mails on the server for 5 days or until i delete them. These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. if i even delete a mail from outlook the server doesn't delete or old mails are not deleted. what can be the problem ? thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394.html Sent from the Dovecot mailing list archive at Nabble.com. From robert at schetterer.org Tue Sep 4 11:30:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 10:30:22 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST Message-ID: <5045BC1E.9040700@schetterer.org> Hi, perhaps somebody wants to this --snip Microsoft? Outlook? 2013 Preview Outlook 2013 Preview implements the IMAP LIST extension specified in [RFC6154] as the XLIST command. ---snip taken out of [MS-STANOIMAP].pdf which zip you may download here http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 11:34:59 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 09:34:59 +0100 (BST) Subject: [Dovecot] problems with ssl cert In-Reply-To: References: Message-ID: <1346747699.13635.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: "cc "maco" young" > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 10:50 > Subject: [Dovecot] problems with ssl cert > > in conf.d/10-ssl.conf: > > ssl = yes > ssl_cert = ssl_key = > when try to set up an account in thrunderbird, get /var/log/mail.log: > > imap-login: Disconnected (no auth attempts): rip=223.205.150.234, > lip=xx.xx.xx.xx, TLS: SSL_read() failed: error:14094416:SSL > routines:SSL3_READ_BYTES:sslv3 alert certificate unknown: SSL alert number > > openssl s_client -connect mail.myserver.com:993 - successful > > using the same cert in gning / chrome is successful, so think the cert is ok > Hi again, Do a fresh Thunderbird installation and get rid of any folders? related to thunderbird on either OS you have installed it on. s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From robert at schetterer.org Tue Sep 4 13:53:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 12:53:33 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045BC1E.9040700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> Message-ID: <5045DDAD.3010705@schetterer.org> Am 04.09.2012 10:30, schrieb Robert Schetterer: > Hi, perhaps somebody wants to this > > --snip > Microsoft? Outlook? 2013 Preview > Outlook 2013 Preview implements the IMAP LIST extension specified in > [RFC6154] as the XLIST command. > ---snip > > taken out of [MS-STANOIMAP].pdf > > which zip you may download here > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > just for info, by small testing Outlook 2013 preview imap specialuse is working with dovecot at minimum for Trash and Sent i have set this mailbox Sent { special_use = \Sent auto=subscribe } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash auto=subscribe } examples are in /etc/dovecot/conf.d/15-mailboxes.conf couldnt test Archive/Drafts/Junk but they "look" Working i had my problems with some new gui functions *g the preview is only in english or spanish so dont know if it will work with other languages then english, lets hope so feel free to test yourself -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 16:31:49 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 14:31:49 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <1346765509.32698.YahooMailNeo@web132201.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 16:00 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 2:31 PM, Spyros Tsiolis wrote: > >> . . . . ?%< . . . . . %< . . . . ?. . . . >? >what you showed is what I remember when I was last hacking in TB. ?now all options are gone. ?screenshot attached > > >think I know problem. ?got tired of TB - no options, no feedback. ?ok if everything works; otherwise sucks. > > >went to claws. ?they have actual messages, actual logs. ?had similar problem, but claws easily allowed me to override. ?see this:?http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2199 > > >know my cert chain ok because?http://www.sslshopper.com/ssl-checker.html checks chain ok > > >anyway, thanks for your help, and goodbye thunderbird > > > > I remember being stuck on this. I know what you are saying. Give it another go. Tinker with it. You'll get there. If you're fed up with TB, that's another thing. However, you _can_ get round this. I know I did. Just my 2c, Regards, spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Sep 4 18:20:08 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 4 Sep 2012 17:20:08 +0200 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <20120904152007.GA20273@PC211.ikt.de> On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: > Hi! [..] > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. http://wiki2.dovecot.org/Tools/Doveadm/Expunge HTH Dennis [..] From alessio at skye.it Tue Sep 4 18:40:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 04 Sep 2012 17:40:33 +0200 Subject: [Dovecot] doveadm command to block a user? In-Reply-To: <50446D34.5000702@um.es> References: <50446D34.5000702@um.es> Message-ID: <504620F1.5040801@skye.it> Il 03/09/2012 10:41, Angel L. Mateo ha scritto: > Hello, > > Is there any command to be able to lock imap access to a specific > user? With doveadm kick I could close his connections, but I want to > avoid future ones too and just for imap. > > Any way to do it? > I can do it by disable specific user at backend level, for example vpopmail can do it, but also with SQL can be simple. With specific query you can disable only IMAP, only POP and so on. In the password query you can insert something like this: WHERE username = '%u' AND active = '1' AND ((IF( %a=110, pop, 0 )) =1 OR (IF( %a=25, smtp , 0 )) =1 OR (IF(( %a = 143 ) AND '%l' = '127.0.0.1', webmail, 0 ) = 1 ) OR ( IF(( %a = 143 ) AND '%l' !='127.0.0.1', imap, 0 ) = 1)) Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From sven at svenhartge.de Tue Sep 4 18:40:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 17:40:17 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <20120904152007.GA20273@PC211.ikt.de> Message-ID: <0937s3tbapv8@mids.svenhartge.de> Dennis Guhl wrote: > On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: >> Hi! >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > http://wiki2.dovecot.org/Tools/Doveadm/Expunge Yes, I know about doveadm expunge. I am searching for a search_query which only matches messages with the flag \Deleted whose flag has been set 24h or longer ago. Doing something like doveadm expunge -A mailbox \* DELETED is not the solution, since it will expunge all deleted mails without looking at the time when they have been deleted. Using savedbefore does not work either, since a message may have been saved 2 years ago but only recently deleted. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Tue Sep 4 19:11:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 19:11:35 +0300 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> On 3.9.2012, at 14.31, Sven Hartge wrote: > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. Dovecot doesn't keep track of that information, so you can't. From sven at svenhartge.de Tue Sep 4 19:35:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 18:35:51 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> Message-ID: <0937vh4hfsv8@mids.svenhartge.de> Timo Sirainen wrote: > On 3.9.2012, at 14.31, Sven Hartge wrote: >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > Dovecot doesn't keep track of that information, so you can't. I suspected this. Would be nice though. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Tue Sep 4 19:37:24 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:37:24 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? Message-ID: <50462E44.2090505@Media-Brokers.com> Almost every message I'm getting through this list is duplicated, down to the same exact message-ID... Anyone else seeing this? Charles From CMarcus at Media-Brokers.com Tue Sep 4 19:40:48 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:40:48 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50462F10.90906@Media-Brokers.com> On 2012-09-04 12:37 PM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Even this one was duplicated... From philip at turmel.org Tue Sep 4 19:58:01 2012 From: philip at turmel.org (Phil Turmel) Date: Tue, 04 Sep 2012 12:58:01 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <50463319.8060309@turmel.org> On 09/04/2012 12:40 PM, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Even this one was duplicated... Not here :-) Phil From stan at hardwarefreak.com Tue Sep 4 20:02:38 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:02:38 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <5046342E.4010808@hardwarefreak.com> On 9/3/2012 11:21 PM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx What does TB activity manager say? -- Stan From stan at hardwarefreak.com Tue Sep 4 20:07:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:07:05 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50463539.9090005@hardwarefreak.com> On 9/4/2012 11:37 AM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Nope. Make any changes to Postfix or your script recently? -- Stan From florob at babelmonkeys.de Tue Sep 4 21:25:54 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 04 Sep 2012 20:25:54 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation Message-ID: <504647B2.5060804@babelmonkeys.de> Hello everyone and Timo in particular, about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present in Dovecot. I had always disliked this a bit, because it replicates a lot of code. This patch generalizes the HMAC function to take a hash_method struct as parameter, and changes existing code which uses the "old" HMAC function to use this new one. I'm not really sure this is actually a good idea, but I still felt I should provide the code in case you would want to merge it upstream. Attached is the patch as a hg export based on the revision of dovecot-2.2 current at the time of writing. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID e2f682fab829c2ef23a050f884191f57e2fb5d60 # Parent 9bc2e718392ceaa327f14b50163232b629cd54d1 lib: Generalize hmac to be hash independent diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,64 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,12 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + ctx->hash = meth; + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,23 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +struct hmac_context { + void *ctx, *ctxo; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From rob0 at gmx.co.uk Tue Sep 4 22:16:50 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 4 Sep 2012 14:16:50 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <20120904191649.GN3672@harrier.slackbuilds.org> On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rplatel at tucows.com Tue Sep 4 23:01:45 2012 From: rplatel at tucows.com (Richard Platel) Date: Tue, 4 Sep 2012 16:01:45 -0400 Subject: [Dovecot] Custom auth process in dovecot 2 Message-ID: Hi, I'm trying to upgrade from dovecot-1.1.x to 2.1.7. We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. In dovecot 1.1 we could configure this with auth external { socket connect { master { path = /var/run/dovecot/auth.sock } } } as per http://wiki.dovecot.org/MainConfig I haven't been able to figure out how to do this in 2.1.7, is it possible? From tss at iki.fi Tue Sep 4 23:05:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 23:05:55 +0300 Subject: [Dovecot] Custom auth process in dovecot 2 In-Reply-To: References: Message-ID: <99C78902-F077-4DB0-991C-39171320EF1E@iki.fi> On 4.9.2012, at 23.01, Richard Platel wrote: > I'm trying to upgrade from dovecot-1.1.x to 2.1.7. > > We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. > > In dovecot 1.1 we could configure this with > > auth external { > socket connect { > master { > path = /var/run/dovecot/auth.sock > } > } > } > > as per http://wiki.dovecot.org/MainConfig > > I haven't been able to figure out how to do this in 2.1.7, is it possible? Possibility a) Disable the regular service auth, something like: service auth { unix_listener login/login { mode = 0 } } and then just create the /var/run/dovecot/login/login socket yourself. You may need/want to do this for other auth-related sockets too. Possibility b) Tell login processes to connect to your socket instead: service imap-login { executable = imap-login newlogin } From CMarcus at Media-Brokers.com Wed Sep 5 00:02:00 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 17:02:00 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50463539.9090005@hardwarefreak.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> Message-ID: <50466C48.1020303@Media-Brokers.com> On 9/4/2012 1:07 PM, Stan Hoeppner wrote: > On 9/4/2012 11:37 AM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Nope. Make any changes to Postfix or your script recently? Nope... but, your reply made me actually take a closer look... It is only happening for emails that are filtered to folders (like my lists)... And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... I'll disable filtering on one, and see if that solves the problem (I expect it will)... Sorry for the noise... -- Best regards, Charles From h.reindl at thelounge.net Wed Sep 5 00:57:43 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 04 Sep 2012 23:57:43 +0200 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50466C48.1020303@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> Message-ID: <50467957.2010602@thelounge.net> Am 04.09.2012 23:02, schrieb Charles Marcus: > On 9/4/2012 1:07 PM, Stan Hoeppner wrote: >> On 9/4/2012 11:37 AM, Charles Marcus wrote: >>> Almost every message I'm getting through this list is duplicated, down >>> to the same exact message-ID... >>> >>> Anyone else seeing this? >> >> Nope. Make any changes to Postfix or your script recently? > > Nope... but, your reply made me actually take a closer look... > > It is only happening for emails that are filtered to folders (like my lists)... > > And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened > another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... > > I'll disable filtering on one, and see if that solves the problem (I expect it will)... better use SIEVE filters on the server for moving list-messages to folders instead the overhead of TB which is in fact copy (download/upload) followed by delete so you have no problems with concurrent clients including mobile devices -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From jtam.home at gmail.com Wed Sep 5 02:02:44 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 4 Sep 2012 16:02:44 -0700 (PDT) Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: References: Message-ID: On Tue, 4 Sep 2012, Sven Hartge wrote: > I am searching for a search_query which only matches messages with the > flag \Deleted whose flag has been set 24h or longer ago. > > Doing something like > > doveadm expunge -A mailbox \* DELETED > > is not the solution, since it will expunge all deleted mails without > looking at the time when they have been deleted. > > Using savedbefore does not work either, since a message may have been > saved 2 years ago but only recently deleted. You could work out a cron job that does a "doveadm search ...", then diff it with one generated 24 hours ago, extract the common GID/UUIDs, then do a second pass to delete them. Not elegant, but it would probably work. (I just read the man page, and there doesn't seem to be an option to pipe the search like "doveadm search ... | doveadm expunge ...". You'll have to expunge one message at a time. Ugh.) Or maybe the advice "don't worry, be happy" applies here. I find that too many "mistake mitigation" features is counterproductive as users habitually relying on them, rather than being careful. Extending your grace period to a week will lessen the probability of this race condition. Joseph Tam From kgc at corp.sonic.net Wed Sep 5 02:06:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 16:06:37 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <5046897D.1050502@corp.sonic.net> On 09/03/12 12:06, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. >> >> There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) > > Did you try stopping both and then starting them again? That clears up all the state they have. I stopped both directors last night and they were able to stay in sync after they were restarted. Could corruption of the in memory state lead to the connections being dropped? If this happens again I'll try to get a tcpdump and an strace so the bug can get squashed. -K From tss at iki.fi Wed Sep 5 03:58:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 03:58:19 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > passdb { > args = proxy=y nopassword=y > driver = static > } I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. From tss at iki.fi Wed Sep 5 04:23:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 04:23:39 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <6CB24D47-CD68-420B-A3E7-5C79DE1CD757@iki.fi> On 5.9.2012, at 3.58, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. Although the extra users should be freed from the memory after 15 minutes. Hmm. Once Dovecot supports moving existing connections from one backend server to another without the client noticing anything, the director could be simplified by using consistent hashing and when the number of backends changes, the director could start moving connections to their proper backends. During this move new connections would be handled by 1) if old backend = new backend just forward the connection there or 2) if they're different, request immediate move for that user's existing connections and wait for it to be finished before letting new connections finish. Or alternatively if the user isn't just being moved at that time, forward the connection to the old server and let it be part of the later move. The main difference here is that directors wouldn't need to keep any track of user -> backend associations. The moving period could still be a bit tricky to handle well, especially since the situation can change again while a previous move is still going on. From kgc at corp.sonic.net Wed Sep 5 07:06:51 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 21:06:51 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <5046CFDB.90809@corp.sonic.net> On 9/4/2012 5:58 PM, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. It doesn't look like there was a higher than normal number of failed logins leading up to the connection issues. I'm going to write some more stats collection tools to track state on the directors and see what comes of it. Can the director proxy validate the username via a unix pw lookup but not check the password? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From p at state-of-mind.de Wed Sep 5 09:14:10 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 08:14:10 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045DDAD.3010705@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> Message-ID: <20120905061409.GC12086@state-of-mind.de> * Robert Schetterer : > Am 04.09.2012 10:30, schrieb Robert Schetterer: > > Hi, perhaps somebody wants to this > > > > --snip > > Microsoft? Outlook? 2013 Preview > > Outlook 2013 Preview implements the IMAP LIST extension specified in > > [RFC6154] as the XLIST command. > > ---snip > > > > taken out of [MS-STANOIMAP].pdf > > > > which zip you may download here > > > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > > > > just for info, by small testing > Outlook 2013 preview imap specialuse is working with dovecot > at minimum for Trash and Sent > > i have set this > > mailbox Sent { > special_use = \Sent > auto=subscribe > } > mailbox "Sent Messages" { > special_use = \Sent > } Why did you set the \Sent folder twice? Setting it once should suffice. > the preview is only in english or spanish > so dont know if it will work with other languages then english, lets hope so Try this: mailbox "Gesendete Objekte" { special_use = \Sent auto=subscribe } AFAIK the English version should automagically map itself to that folder. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From duihi77 at gmail.com Wed Sep 5 09:45:09 2012 From: duihi77 at gmail.com (Duane Hill) Date: Wed, 5 Sep 2012 06:45:09 +0000 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <723005990.20120905064509@gmail.com> On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > * Robert Schetterer : >> Am 04.09.2012 10:30, schrieb Robert Schetterer: >> > Hi, perhaps somebody wants to this >> > >> > --snip >> > Microsoft? Outlook? 2013 Preview >> > Outlook 2013 Preview implements the IMAP LIST extension specified in >> > [RFC6154] as the XLIST command. >> > ---snip >> > >> > taken out of [MS-STANOIMAP].pdf >> > >> > which zip you may download here >> > >> > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >> > >> >> just for info, by small testing >> Outlook 2013 preview imap specialuse is working with dovecot >> at minimum for Trash and Sent >> >> i have set this >> >> mailbox Sent { >> special_use = \Sent >> auto=subscribe >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } > Why did you set the \Sent folder twice? Setting it once should suffice. FYI: Default config shows it listed twice in: conf.d/15-mailboxes.conf ... # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } ... -- If at first you don't succeed... ...so much for skydiving. From robert at schetterer.org Wed Sep 5 10:18:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:18:14 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <5046FCB6.7080700@schetterer.org> Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > Try this: > > mailbox "Gesendete Objekte" { > special_use = \Sent > auto=subscribe > } i will do when Outlook 2013 in german got released > > AFAIK the English version should automagically map itself to that folder. agree ,it should > > p at rick thunderbirds status about xlist etc can be seen here https://bugzilla.mozilla.org/show_bug.cgi?id=558659 looks like , its on the road, from pure tec side no idea if and when it will go released -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 5 10:28:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:28:53 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <723005990.20120905064509@gmail.com> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <723005990.20120905064509@gmail.com> Message-ID: <5046FF35.4070504@schetterer.org> Am 05.09.2012 08:45, schrieb Duane Hill: > On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > >> * Robert Schetterer : >>> Am 04.09.2012 10:30, schrieb Robert Schetterer: >>>> Hi, perhaps somebody wants to this >>>> >>>> --snip >>>> Microsoft? Outlook? 2013 Preview >>>> Outlook 2013 Preview implements the IMAP LIST extension specified in >>>> [RFC6154] as the XLIST command. >>>> ---snip >>>> >>>> taken out of [MS-STANOIMAP].pdf >>>> >>>> which zip you may download here >>>> >>>> http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >>>> >>> >>> just for info, by small testing >>> Outlook 2013 preview imap specialuse is working with dovecot >>> at minimum for Trash and Sent >>> >>> i have set this >>> >>> mailbox Sent { >>> special_use = \Sent >>> auto=subscribe >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } > >> Why did you set the \Sent folder twice? Setting it once should suffice. > > FYI: Default config shows it listed twice in: > > conf.d/15-mailboxes.conf > ... > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > ... > "Sent Messages" was/is used in some clients for Sent, guess its also in the example, looks like the new client versions with xlist etc changed this sometimes,too ( going directly Sent , but i didnt investigated hardly in this, test yourself), so it may not longer needed in the future, meanwhile it does not hurt somenone, since yet But i agree feature need more real world examples and explains in some wiki related to div mail clients in their different versions guess this is on the road -- Best Regards MfG Robert Schetterer From dafan.zhai at securepoint.de Wed Sep 5 11:53:37 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Wed, 05 Sep 2012 10:53:37 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder Message-ID: <50471311.2050002@securepoint.de> Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, &metadata); but Valgrind finds a memory leak when this function is called: ---------------------------------------------------snip-------------------------------------------------------------- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304== by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---------------------------------------------------snip-------------------------------------------------------------- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---------------------------------------------------snip-------------------------------------------------------------- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---------------------------------------------------snip-------------------------------------------------------------- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1 branch fixing memory leaks: http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b It would be nice if you could give me some hints how I can address this issue? Thanks in advance! Greetings, Dafan From lgb at lgb.hu Wed Sep 5 13:20:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 5 Sep 2012 12:20:26 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? Message-ID: <20120905102026.GB23439@vega.lgb.hu> Dear All, I have a bit complex question about ldap based user/passdb with some twists caused by the LDAP schema I have to use. It's complex for me at least, since it's my first time to try to setup any dovecot install which is more than a "trivial" thing eg for some unix system users. What I'd like to do: Users are stored in LDAP with the following schema: storageMailUid: contains user's uid in x at z format, and it CAN be different than mail address. This value is used to login (pop3/imap) and to get the maildir path (well, home) in the form of /mailstorage/z/x/ regardless of the value of the "mail" attribute. If @z part is missing (I mean the user supplied login, not the LDAP!), some default value (example.com) should be treated (I guess that's easy: auth_default_realm may be enough, isn't it). It is NOT possible to login via pop3/imap with mail, only with storageMailUid! This storageMailUid can specify a string which is not even an existing mail address, of course. mail: contains user's mail address, or even mail addresses (there can be more mail addresses - more mail ldap attributes - for a single ldap entry). This attribute cannot be used for pop3/imap login, neither it counts to resolve the maildir path, it only counts when a mail is received: the location of the user's maildir must be got from storageMailUid. storageMailQuota: it contains the quota value (in bytes) for the given user, which should be enforced on receiving mail, or by using IMAP (not counting the Trash folder which is a fixed size for all users and it must be handled outside of this). All user maildirs have the same fixed unix UID/GID, which is not stored in LDAP, but must be configured statically. "Home directory" of the user is also not stored in LDAP, it must be got from storageMailUid LDAP parameter. Mails are received via LMTP only. I'd like to use prefetchdb to minimalize the amount of LDAP lookups (however I can live without that). I am unsure if auth_bind is OK, or it should not be used, currently I'd like to play with auth_bind, since it worked before on other servers well. I'm totally lost with the user_attrs/pass_attrs to create this kind of configuration. In case of static userdb, it was easy to set up: passdb { args = /etc/dovecot/dovecot-ldap-passdb.conf driver = ldap } userdb { args = uid=vmail gid=vmail home=/mailstorage/%Ld/%Ln driver = static } With this, pop3/imap worked (quota was not of the scope yet, though), however lmtp is not ("passdb doesn't support lookups, can't verify user's existence" which is odd for me, as userdb does not support it in my opinion). Please, give some suggestion how to set up userdb and passdb to support this configuration which works with the described scenario. I should also set iterate_filter and iterate_attrs I guess, so some doveadm commands can work then (-A stuffs, I guess). I am also not sure if auth service should be used or not (I mean extra configuration related): since I want LMTP not LDA, I guessed I don't need it, but I am not sure. I am also confused, because on receiving a mail (via LMTP) different kind of LDAP lookup is needed: then mail must be searched, but it's storageMailUid based lookup in case of pop3 or imap login ... Is it possible to give different userdb/passdb for lmtp and pop3/imap? In theory it's even possible to have x at example.com as mail and y at example.com as storageMailUid for one user, and the opposite for the another, so lookups cannot be done together for mail and storageMailUid. The LDAP schema/rest of the system works this way, not an option to change. Any help is greatly welcomed. Thanks a lot in advance, G?bor From bangkokmaco at gmail.com Wed Sep 5 13:23:25 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 17:23:25 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <5046342E.4010808@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner wrote: > On 9/3/2012 11:21 PM, cc young wrote: > > cannot get TB to recognize either pop3/s or imap/s server > > > > can connect just fine with: > > > > openssl s_client -connect ms1.myserver.net:993 > > . login ... > > > > but trying with TB /var/log/mail.log gets: > > > > dovecot: pop3-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > dovecot: imap-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > What does TB activity manager say? > > Activity Manager is blank as an update, went to Claws email client. it had enough feedback and gave me enough ways to tweak - got everything going. went back to TB. recognized server, but no username/password. perhaps it's not letting the full email - which is the user name - through. but who knows? no feedback. From CMarcus at Media-Brokers.com Wed Sep 5 13:58:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 06:58:02 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50467957.2010602@thelounge.net> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> Message-ID: <5047303A.5060609@Media-Brokers.com> On 2012-09-04 5:57 PM, Reindl Harald wrote: > better use SIEVE filters on the server I know, it is on my ToDo list... we only just recently migrated this server to Dovecot, and I've had my plate full with other issues, which are now mostly resolved, so I'm about ready to circle back and finish up (installing SOGo, enabling sieve, etc), but I need to spin up a new Gentoo VM first, because I want to get the current server moved over to our new VMWare environment. -- Best regards, Charles From keijser at stone-it.com Wed Sep 5 14:00:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 13:00:04 +0200 Subject: [Dovecot] managesieve problem Message-ID: <1346842804.29847.13.camel@localhost> Hi, I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) and pigeonhole (2.0.9). All works well except sieve. I've already posted on the roundcube forum (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was told I should ask here since it's more likely a dovecot issue rather than roundcube. The problem is that if I want to manage sieve filters from roundcube, I get this error in maillog: Sep 5 12:55:39 emperor dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=1575, secured Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage The managesieve daemon is listening though: emperor ~ # netstat -tlpne | grep dovecot tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 0 1275042 362/dovecot tcp 0 0 0.0.0.0:2000 0.0.0.0:* LISTEN 0 1275037 362/dovecot tcp 0 0 0.0.0.0:4190 0.0.0.0:* LISTEN 0 1275035 362/dovecot tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN 0 1275044 362/dovecot tcp 0 0 :::143 :::* LISTEN 0 1275043 362/dovecot tcp 0 0 :::2000 :::* LISTEN 0 1275038 362/dovecot tcp 0 0 :::4190 :::* LISTEN 0 1275036 362/dovecot tcp 0 0 :::993 :::* LISTEN 0 1275045 362/dovecot emperor ~ # telnet localhost 4190 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus" "NOTIFY" "mailto" "SASL" "PLAIN" "STARTTLS" "VERSION" "1.0" OK "Dovecot ready." Before I paste any configuration files, I wanted to ask first if this is a common/known issue. Else, please let me know what additional information I should provide. Thanks in advance! kind regards, L?on From CMarcus at Media-Brokers.com Wed Sep 5 14:02:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 07:02:32 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: <50473148.8050905@Media-Brokers.com> On 2012-09-05 6:23 AM, cc maco young wrote: > On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >> What does TB activity manager say? > Activity Manager is blank > > as an update, went to Claws email client. it had enough feedback and gave > me enough ways to tweak - got everything going. > > went back to TB. recognized server, but no username/password. perhaps > it's not letting the full email - which is the user name - through. but > who knows? no feedback. You're looking at the wrong end. You need to be looking at your SERVER logs, not the Clients. -- Best regards, Charles From stephan at rename-it.nl Wed Sep 5 14:16:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 13:16:19 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346842804.29847.13.camel@localhost> References: <1346842804.29847.13.camel@localhost> Message-ID: <50473483.9020702@rename-it.nl> Op 9/5/2012 1:00 PM, L?on Keijser schreef: > Hi, > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > and pigeonhole (2.0.9). All works well except sieve. I've already posted > on the roundcube forum > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > told I should ask here since it's more likely a dovecot issue rather > than roundcube. [...] > Before I paste any configuration files, I wanted to ask first if this is > a common/known issue. Else, please let me know what additional > information I should provide. The issue does look familiar and most likely it is already solved. But, the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. Upgrade first. Regards, Stephan. From bangkokmaco at gmail.com Wed Sep 5 14:18:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 18:18:40 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: On Wed, Sep 5, 2012 at 6:02 PM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: > >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner >> >wrote: >> >>> What does TB activity manager say? >>> >> > Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. >> > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. > > yes, you are right. will get back. From amateo at um.es Wed Sep 5 14:40:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 05 Sep 2012 13:40:54 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) Message-ID: <50473A46.1080807@um.es> Hello, We have a dovecot servers farm with storage in NFS and indexes in local files. Under heavy load conditions we a log of messages in the form: Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. PS: I have attached my dovecot -n output -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_privileged_group = mail mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = maildir:User quota quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es service anvil { client_limit = 2051 } service auth { client_limit = 3072 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From tss at iki.fi Wed Sep 5 14:50:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 14:50:38 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50473A46.1080807@um.es> References: <50473A46.1080807@um.es> Message-ID: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> On 5.9.2012, at 14.40, Angel L. Mateo wrote: > Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) > > We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). > > So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). From zucca at systemschmiede.com Wed Sep 5 14:59:34 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Wed, 05 Sep 2012 13:59:34 +0200 Subject: [Dovecot] strange Folder in subscription list (was: Imap Ghost Folder) Message-ID: <50473EA6.3020203@systemschmiede.com> Hi list, here's the complete Problem for easy reading again. Can someone reproduce this? I have a strange "ghostfolder" when using shares/%%n/ as prefix for shared namespace. If i swith only this setting to shares/%%u/ everything is just as wanted, only then the prefix is very long, so i would prefer the name. dovecot --version 2.1.7 dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-14-pve i686 Debian 6.0.5 simfs auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%%d/%%n/Maildir mail_plugins = acl mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u prefix = shares/%%n/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } ssl_cert = took seconds) In-Reply-To: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> Message-ID: <50474093.3010306@um.es> El 05/09/12 13:50, Timo Sirainen escribi?: > On 5.9.2012, at 14.40, Angel L. Mateo wrote: > >> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >> >> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >> >> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. > > That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). > Ok, I'll try this. But... is there any way to know how a readdir would be done? From keijser at stone-it.com Wed Sep 5 15:49:39 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 14:49:39 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <50473483.9020702@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> Message-ID: <1346849379.4102.3.camel@localhost> Hi Stephan, On Wed, 2012-09-05 at 13:16 +0200, Stephan Bosch wrote: > Op 9/5/2012 1:00 PM, L?on Keijser schreef: > > Hi, > > > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > > and pigeonhole (2.0.9). All works well except sieve. I've already posted > > on the roundcube forum > > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > > told I should ask here since it's more likely a dovecot issue rather > > than roundcube. > [...] > > Before I paste any configuration files, I wanted to ask first if this is > > a common/known issue. Else, please let me know what additional > > information I should provide. > > The issue does look familiar and most likely it is already solved. But, > the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. > > Upgrade first. Upgraded to 2.0.21. Now I get this error: Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory for substitition in storage root directory (sieve_dir=~/sieve) Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage. regards, L?on From keijser at stone-it.com Wed Sep 5 16:26:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 15:26:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346849379.4102.3.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> Message-ID: <1346851564.4102.7.camel@localhost> On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: > Upgraded to 2.0.21. Now I get this error: > > > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: > sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory > for substitition in storage root directory (sieve_dir=~/sieve) > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: > Failed to open Sieve storage. I should probably mention that I have virtual domains/users, all located in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a real user on the box. regards, L?on From stephan at rename-it.nl Wed Sep 5 16:29:46 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 15:29:46 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346851564.4102.7.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> Message-ID: <504753CA.3060609@rename-it.nl> Op 9/5/2012 3:26 PM, L?on Keijser schreef: > On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: >> Upgraded to 2.0.21. Now I get this error: >> >> >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: >> sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory >> for substitition in storage root directory (sieve_dir=~/sieve) >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: >> Failed to open Sieve storage. > > I should probably mention that I have virtual domains/users, all located > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > real user on the box. This should provide all the info you need on that: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Stephan. From p at state-of-mind.de Wed Sep 5 19:22:21 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 18:22:21 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5046FCB6.7080700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <5046FCB6.7080700@schetterer.org> Message-ID: <20120905162219.GB24756@state-of-mind.de> * Robert Schetterer : > Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > > Try this: > > > > mailbox "Gesendete Objekte" { > > special_use = \Sent > > auto=subscribe > > } > > i will do when Outlook 2013 in german got released > > > > AFAIK the English version should automagically map itself to that folder. > > agree ,it should > > > > > p at rick > > thunderbirds status about xlist etc can be seen here > > https://bugzilla.mozilla.org/show_bug.cgi?id=558659 > > looks like , its on the road, from pure tec side > no idea if and when it will go released It is implemented. We had to adapt a few extra functions to deal with 64 bit stuff. Currently it is not being pushed further because of the uncertainty of TBs future. Once it will become clear how new features will be released we will spend the rest of money and time to ship the feature. For now I will not spend a single more Euro. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From CMarcus at Media-Brokers.com Thu Sep 6 00:03:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 17:03:27 -0400 Subject: [Dovecot] mail_location / mail_home In-Reply-To: <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> References: <38719C76-8672-4F85-B4EA-8802CF85BE63@4lin.net> <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> Message-ID: <5047BE1F.2000103@Media-Brokers.com> On 2012-08-31 4:15 AM, Denny Schierz wrote: > I think, it doesn't make any difference, if I have "~" or a fixed path > :-/ On the productive system (also 2.1.9) I get messages like: [...] > dovecot.lda-dupes/tmp) failed: Not a directory [...] So, I need a way, > to move ~11.000 Maildirs .... any suggestions? mmv is a godsend... http://linux.dsplabs.com.au/mmv-copy-append-link-move-multiple-files-under-linux-shell-bash-by-wildcard-patterns-p5/ -- Best regards, Charles From ben at indietorrent.org Thu Sep 6 01:20:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 05 Sep 2012 18:20:57 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot Message-ID: <5047D049.8080004@indietorrent.org> Hello, I am attempting to use the Antispam plug-in for Dovecot and am having trouble with the implementation. My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. Mail that SpamAssassin tags as SPAM is moved into the affected user's "Junk" folder (/var/vmail/example.com/user/Maildir/.Junk). I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam . After restarting Dovecot, my IMAP client (Thunderbird v15) reports the following whenever I attempt to move mail from Inbox to "Junk" directory: "[CANNOT] antispam plugin not configured" That exact phrase appears only once in a Google search ( http://dovecot-antispam.sourcearchive.com/documentation/1.1plus-p20090218.git.g28075fa/mailtrain_8c-source.html ): if (!hamaddr || !spamaddr) { mail_storage_set_error(t->box->storage, ME(NOTPOSSIBLE) "antispam plugin not configured"); return -1; } Based on the surrounding source code, which I realize may be old, it seems that Ham and Spam addresses are required, or similar. I am using the following configuration options: /etc/dovecot/dovecot.conf -------------------- ## IMAP specific settings protocol imap { mail_executable = /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_plugins = quota imap_quota antispam } # [...] plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_spam_pattern_ignorecase = spam;junk } -------------------- For what it's worth, the logging directives seem not to have any effect (and perhaps the rest of the directives do not, either, hence the problem I'm having). I tried changing the above directives to their older directive names (I also changed the order slightly, which may be irrelevant): -------------------- plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_spam_pattern_ignorecase = spam;junk } -------------------- This changed the message in my IMAP client, when attempting to move a message from my Inbox to the Junk folder, to: "[SERVERBUG] failed to send mail" Does this indicate a problem with the "/usr/bin/sa-learn-pipe.sh" script that is provided on the Wiki (cited above)? Also, it was not clear from the documentation ( http://johannes.sipsolutions.net/files/antispam.html ) whether or not the Pipe back-end requires the X-DSPAM-Signature header, and if so, to what value it should be set when using SpamAssassin. Thanks for any pointers, -Ben From jtam.home at gmail.com Thu Sep 6 01:22:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 5 Sep 2012 15:22:31 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: Message-ID: On Wed, 5 Sep 2012, Charles Marcus wrote > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. Actually, I think he did supply that and it showed imap and pop3 not getting any auth information. > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx There are other log options to verbosely log auth and ssl information, and that might help. I frequently see the above log messages when the client and the server get their security protocol (SSL/TLS) and ports crossed up (i.e. trying IMAP/SSL on port 143 or trying IMAP/TLS on port 993). Try fiddling with TB's setting here and see if anything shakes loose. You can also use openssl and simulate the server end and maybe that will shed light on the situation: (You'll have to read the man page yourself to find all the options) openssl s_server -accept 993 ... Joseph Tam From kgc at corp.sonic.net Thu Sep 6 03:17:46 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 05 Sep 2012 17:17:46 -0700 Subject: [Dovecot] dovecot stats: useful data to gather In-Reply-To: <20120603001049.GA10970@daniel.localdomain> References: <20120601205839.GG2176@state-of-mind.de> <20120603001049.GA10970@daniel.localdomain> Message-ID: <5047EBAA.8090500@corp.sonic.net> On 06/02/12 17:10, Daniel Parthey wrote: > Patrick Ben Koetter wrote: >> following our discussion on dovecot stats at the LinuxTag 2012 my team and I >> sat down and put together a list of stat items we think to be useful in daily >> dovecot usage. >> >> Besides pulling together all the data we also think it would be useful to have >> an SNMP interface to access the stats. Our offer to create and contribute a >> standalone web interface for dovecot stats stands. > > This should be done via SNMP subagent, but how could you differentiate > different dovecot instances on the same machine, different snmp ports > for the subagent, or different snmp trees? I'd suggest some additional performance metrics like min/max/avg time to authenicate, establish a proxy session and perhaps include auth failure causes counters as well. I personally wouldn't want to see this implemented as an SNMP subagent but so long as the stats would be available off a local socket directly I think everyone would be happy. -K From stan at hardwarefreak.com Thu Sep 6 05:36:02 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 05 Sep 2012 21:36:02 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: <50480C12.5040500@hardwarefreak.com> On 9/5/2012 6:02 AM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan >> Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. He'd already checked the server logs, posted some of them, and found no useful information. Checking the client log is the next logical step, whether it turns out to contain useful information or not. In a client/server application, instructing someone to only check half the logs is bad advice Charles. Useful troubleshooting information can be found in either, or both, depending on the circumstances and operation that's failing. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 6 05:50:00 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 6 Sep 2012 04:50:00 +0200 Subject: [Dovecot] Length of attachment filenames In-Reply-To: References: Message-ID: <20120906025000.GA11887@daniel.localdomain> Fuhrmann, Hauke wrote: > I have a beginners question to Dovecot: does it have a restriction on > the supported length and character encodings of filenames of > attachments? > > Can filenames be arbitrary large? The maximum filename length depends largely on the file system: http://en.wikipedia.org/wiki/Comparison_of_file_systems Regards Daniel -- https://plus.google.com/103021802792276734820 From keijser at stone-it.com Thu Sep 6 09:25:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Thu, 06 Sep 2012 08:25:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <504753CA.3060609@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> <504753CA.3060609@rename-it.nl> Message-ID: <1346912704.32331.2.camel@localhost> On Wed, 2012-09-05 at 15:29 +0200, Stephan Bosch wrote: > > I should probably mention that I have virtual domains/users, all located > > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > > real user on the box. > > This should provide all the info you need on that: > > http://wiki2.dovecot.org/VirtualUsers/Home Works like a charm now, thanks! :) regards, L?on From lists at wildgooses.com Thu Sep 6 09:27:47 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 07:27:47 +0100 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <5047303A.5060609@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> <5047303A.5060609@Media-Brokers.com> Message-ID: <50484263.5040208@wildgooses.com> On 05/09/2012 11:58, Charles Marcus wrote: > I know, it is on my ToDo list... we only just recently migrated this > server to Dovecot, and I've had my plate full with other issues, which > are now mostly resolved, so I'm about ready to circle back and finish > up (installing SOGo, enabling sieve, etc), I have recently noticed owncloud (even has an ebuild for it). Have you re-evaluated roundcube+owncloud vs SOGo for a dav calender/contacts solution? Ed From CMarcus at Media-Brokers.com Thu Sep 6 13:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:10:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5047D049.8080004@indietorrent.org> References: <5047D049.8080004@indietorrent.org> Message-ID: <50487684.8060404@Media-Brokers.com> On 2012-09-05 6:20 PM, Ben Johnson wrote: > My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. 1.2.9 is really old... you really need to upgrade to a recent/stable version. > I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam That is for version 2.x (note the wiki2)... I recommend going ahead and upgrading to the latest 2.1.9 and starting over. -- Best regards, Charles From CMarcus at Media-Brokers.com Thu Sep 6 13:17:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:17:21 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50480C12.5040500@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> <50480C12.5040500@hardwarefreak.com> Message-ID: <50487831.3060103@Media-Brokers.com> On 2012-09-05 10:36 PM, Stan Hoeppner wrote: > He'd already checked the server logs, posted some of them, and found > no useful information. Checking the client log is the next logical > step, whether it turns out to contain useful information or not. In a > client/server application, instructing someone to only check half the > logs is bad advice Charles. Useful troubleshooting information can be > found in either, or both, depending on the circumstances and operation > that's failing. That's what I get for replying in the middle of a thread without reading the whole thing... sorry... Been really busy at $dayjob, and haven't had time to read the lists I'm on for a couple of weeks now. And I didn't advise him to *only* check half the logs, I thought he was *starting out* with the client logs. Of course you're correct that once he'd determined that no auth attempts were happening, checking the client side is the next logical step. But then someone else suggested that it was just because Thunderbird was crapware, which is just plain stupid. If Thunderbird isn't AUTH'ing, it is a config (thus, user) error. -- Best regards, Charles From lgb at lgb.hu Thu Sep 6 14:06:34 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Thu, 6 Sep 2012 13:06:34 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? In-Reply-To: <20120905102026.GB23439@vega.lgb.hu> References: <20120905102026.GB23439@vega.lgb.hu> Message-ID: <20120906110634.GA12495@vega.lgb.hu> Hi Again, On Wed, Sep 05, 2012 at 12:20:26PM +0200, G?bor L?n?rt wrote: [...] > I am also confused, because on receiving a mail (via LMTP) different > kind of LDAP lookup is needed: then mail must be searched, but it's > storageMailUid based lookup in case of pop3 or imap login ... Is it > possible to give different userdb/passdb for lmtp and pop3/imap? Ok, after some "serious" google usage, I found a message in the mail list archive, that it helps to put userdb/passdb section inside the protocol specific part of the configuration. So I have something now like this: protocol pop3 { [...] } protocol imap { [...] } protocol lmtp { [...] passdb { [...] } userdb { [...] } } passdb { [...] } userdb { [...] } I have the idea, that in this way, lmtp should use dbs specified in the lmtp specific protocol settings, any other stuffs will use the settings at the "root level" of the configuration (this also includes iteration specific filter). However eg if I try to deliver a mail through lmtp in this way to a non-existing mail user, I can see in the logs, that lmtp after trying the specific dbs, it will also try the "global" ones, which is not good for me, as it can cause mis-deliveries instead of rejecting (as I have "some at thing" formatted %u for both of uid and mail but often they are not the same for the same user). I would be able to put db specifications into pop3 and imap (so not "global" configuration for them) but I guess in this case other services may (?) fail, like doveadm stuffs with CLI swtich -A (or am I wrong here?). Also it's not as nice, since then I must duplicate the same db specifications in both of imap and pop3 protocol specification parts of the configuration even they are the very same. Is there any idea to fix this little problem and/or a suggestion to solve my problem in a more elegant way than I tried to do? Thanks a lot in advance. - G?bor From dafan.zhai at securepoint.de Thu Sep 6 19:19:27 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Thu, 06 Sep 2012 18:19:27 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <50471311.2050002@securepoint.de> References: <50471311.2050002@securepoint.de> Message-ID: <5048CD0F.9030809@securepoint.de> Hi I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Dafan On 09/05/2012 10:53 AM, Dafan Zhai wrote: > Hi everyone, > > I am writing a dovecot statistic plugin, which calls the > 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as > the 2nd parameter. > > enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; > struct mailbox_metadata metadata; > mailbox_get_metadata(mailbox, metadata_items, &metadata); > > but Valgrind finds a memory leak when this function is called: > ---------------------------------------------------snip-------------------------------------------------------------- > > ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 > of 76 > ==10304== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) > ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) > ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) > ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) > ==10304== by 0x4067102: maildir_mailbox_get_metadata > (maildir-storage.c:486) > ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) > ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) > ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) > ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) > ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) > ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) > ---------------------------------------------------snip-------------------------------------------------------------- > > > I have looked into the source code, and found that from the > 'mail_search_build_init' function call in > lib-storage/index/index-status.c:200 a pool is created, but the pool is > not freed in the mailbox_search_deinit function call in > lib-storage/index/index-status.c:218. > This may be the reason of the memory leak. But I do not know how to free > the pool. > > It seems to me that doveadm is having the same memory leak because it is > using 'mail_search_build_init' too. > ---------------------------------------------------snip-------------------------------------------------------------- > > # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize > INBOX > ==10457== Memcheck, a memory error detector > ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. > ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info > ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX > ==10457== > INBOX vsize=41643319 > ==10457== > ==10457== HEAP SUMMARY: > ==10457== in use at exit: 4,356 bytes in 3 blocks > ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated > ==10457== > ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 > ==10457== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x41AF16C: pool_alloconly_create (in > /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x40DC07D: mail_search_build_init (in > /usr/lib/dovecot/libdovecot-storage.so.0.0.0) > ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in > /usr/bin/doveadm) > ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) > ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) > ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) > ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) > ==10457== by 0x805B863: main (in /usr/bin/doveadm) > ==10457== > ==10457== LEAK SUMMARY: > ==10457== definitely lost: 4,096 bytes in 1 blocks > ==10457== indirectly lost: 0 bytes in 0 blocks > ==10457== possibly lost: 0 bytes in 0 blocks > ==10457== still reachable: 260 bytes in 2 blocks > ==10457== suppressed: 0 bytes in 0 blocks > ==10457== Reachable blocks (those to which a pointer was found) are not > shown. > ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes > ==10457== > ==10457== For counts of detected and suppressed errors, rerun with: -v > ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) > ---------------------------------------------------snip-------------------------------------------------------------- > > > Dovecot version: 2.1.9 > OS: Linux 3.0.30-dist i686 > maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX > > I have applied the following patches from the dovecot 2.1 branch fixing > memory leaks: > http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 > http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b > > > It would be nice if you could give me some hints how I can address this > issue? > > Thanks in advance! > > Greetings, > Dafan -- Follow us on Facebook: Follow us on Twitter: --------------------------------------------------------------------- Securepoint GmbH | Entwicklung Dafan Zhai Salzstr. 1 D-21335 Lueneburg http://www.securepoint.de Tele: ++49 4131 2401-0 Fax: ++49 4131 2401-50 Lueneburg HRB 1776 --------------------------------------------------------------------- CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. GEHEIMHALTUNGSPFLICHT : Dieses E-Mail und alle damit verbundenen Anlagen sind vertraulich und d?rfen nur bestimmten Personen zug?nglich gemacht werden. Sofern Sie nicht zu den angegebenen Empf?ngern geh?ren, benachrichtigen Sie bitte unverz?glich den Absender. Der Inhalt darf weder an Dritte weitergegeben noch zu anderen Zwecken verwendet werden. Die Informationen d?rfen auch nicht auf einem Datentr?ger gespeichert oder auf einen Datentr?ger kopiert werden. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1.9-lib-storage-virtual_size_add_new-memory-leak.patch Type: text/x-patch Size: 873 bytes Desc: not available URL: From ben at indietorrent.org Thu Sep 6 20:56:58 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 13:56:58 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <50487684.8060404@Media-Brokers.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> Message-ID: <5048E3EA.80009@indietorrent.org> On 9/6/2012 6:10 AM, Charles Marcus wrote: > On 2012-09-05 6:20 PM, Ben Johnson wrote: > >> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu > 10.04. > > 1.2.9 is really old... you really need to upgrade to a recent/stable > version. Thanks, Charles. I do see your point. One of the challenges we face in this regard is that we're using a Long-Term-Support version of Ubuntu (10.04) and 1.2.9 is the latest package in the OS's repository. That said, we could upgrade manually, but this is a production server on which downtime must be minimized, and we all know how unexpected issues arise during installation (even when the procedure is tested in a closely equivalent development environment). >> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam > > That is for version 2.x (note the wiki2)... Noted; thank you. > I recommend going ahead and upgrading to the latest 2.1.9 and starting > over. > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle with this further. Thanks again, -Ben From lists at wildgooses.com Thu Sep 6 21:19:03 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 19:19:03 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <5048E917.6030004@wildgooses.com> On 06/09/2012 18:56, Ben Johnson wrote: > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). I personally use (lightweight) virtualisation on any new machine, I really don't see any reason why NOT to. I would typically also setup my mounts such that the operating system is separate from "the data". This makes it easy to upgrade the OS/services, but without touching the data (test before/after on the same data for example) So in my situation I would boot a fairly small (gentoo in my case) virtual environment that runs only dovecot + postfix, it mounts the mail spools separately - I say "boot", but because I'm using linux-vservers, it's really a fancy chroot, and so the instance will start in 2-3 seconds (restarts are similarly near instant). I would upgrade by cloning this installation, upgrading it, testing it to bits, and then to make it live basically you swap this "machine" for the live machine. There are various ways it could be made near seamless, but in my situation I can bear a couple of seconds whilst I literally restart the "machine" Similarly I segregate all my services into a dozen or so "virtual machines", so DNS has it's own "machine" and so does logging, databases, almost every webservice gets its own virtual environment, etc. You could use a full blown vmware/kvm/etc if that floats your boat better, but the point remains it's so trivial to install, makes upgrades to trivial and massively decreases your downtime risk that it's very hard to find a reason NOT to do it... I haven't tried too hard to keep my instances tiny, so each is probably around 400-600MB in my case. However, if it were important this could easily be reduced to 10-100s MB each using various hardlink features. As you can see it's easy to snapshot a whole machine to manage upgrades/backups, etc This is more about infrastructure, but I honestly can't get over how many people are sitting on their hands shackled by "I'm on Debian xxx and I can't install any software newer than 5 years old"... It's so easy to escape from that trap...!! Good luck Ed W From ben at indietorrent.org Thu Sep 6 22:38:07 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 15:38:07 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E917.6030004@wildgooses.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <5048E917.6030004@wildgooses.com> Message-ID: <5048FB9F.7030104@indietorrent.org> On 9/6/2012 2:19 PM, Ed W wrote: > On 06/09/2012 18:56, Ben Johnson wrote: >> >> On 9/6/2012 6:10 AM, Charles Marcus wrote: >>> On 2012-09-05 6:20 PM, Ben Johnson wrote: >>> >>>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >>> 10.04. >>> >>> 1.2.9 is really old... you really need to upgrade to a recent/stable >>> version. >> Thanks, Charles. I do see your point. One of the challenges we face in >> this regard is that we're using a Long-Term-Support version of Ubuntu >> (10.04) and 1.2.9 is the latest package in the OS's repository. >> >> That said, we could upgrade manually, but this is a production server on >> which downtime must be minimized, and we all know how unexpected issues >> arise during installation (even when the procedure is tested in a >> closely equivalent development environment). > > I personally use (lightweight) virtualisation on any new machine, I > really don't see any reason why NOT to. I would typically also setup my > mounts such that the operating system is separate from "the data". This > makes it easy to upgrade the OS/services, but without touching the data > (test before/after on the same data for example) Thanks for your valuable insights, Ed. That seems like a worthwhile approach. > So in my situation I would boot a fairly small (gentoo in my case) > virtual environment that runs only dovecot + postfix, it mounts the mail > spools separately - I say "boot", but because I'm using linux-vservers, > it's really a fancy chroot, and so the instance will start in 2-3 > seconds (restarts are similarly near instant). I would upgrade by > cloning this installation, upgrading it, testing it to bits, and then to > make it live basically you swap this "machine" for the live machine. > There are various ways it could be made near seamless, but in my > situation I can bear a couple of seconds whilst I literally restart the > "machine" > > Similarly I segregate all my services into a dozen or so "virtual > machines", so DNS has it's own "machine" and so does logging, databases, > almost every webservice gets its own virtual environment, etc. You could > use a full blown vmware/kvm/etc if that floats your boat better, but the > point remains it's so trivial to install, makes upgrades to trivial and > massively decreases your downtime risk that it's very hard to find a > reason NOT to do it... While I'm with you here, and I understand the theory (and practice, to some extent), doesn't all of this require a true, physical machine? We can't justify the expense associated with a physical machine in a hosted environment, so we're left with so-called VPSs. My understanding is that OpenVZ cannot be installed on a VPS (for seemingly obvious reasons -- namely, that the VPS is itself an OpenVZ container). > I haven't tried too hard to keep my instances tiny, so each is probably > around 400-600MB in my case. However, if it were important this could > easily be reduced to 10-100s MB each using various hardlink features. > As you can see it's easy to snapshot a whole machine to manage > upgrades/backups, etc > > > This is more about infrastructure, but I honestly can't get over how > many people are sitting on their hands shackled by "I'm on Debian xxx > and I can't install any software newer than 5 years old"... It's so easy > to escape from that trap...!! Perhaps easy, but not necessarily inexpensive. ;-) Thanks again for sharing the details of your strategy; I'll bear all of this in mind moving forward. > Good luck > > Ed W > > -Ben From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 01:12:58 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 00:12:58 +0200 Subject: [Dovecot] dovecotadm error In-Reply-To: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> References: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> Message-ID: <20120906221258.GA9219@daniel.localdomain> Hi Michael, Michael wrote: > doveadm expunge -A mailbox Junk savedbefore 28d > doveadm(root): Error: User listing returned failure > > The mail log file says: > dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table > 'system.users' doesn't exist (using built-in default iterate_query: > SELECT username, domain FROM users) > > dovecot-sql.conf: > iterate_query = SELECT username AS user FROM accounts Some examples in the wiki refer to "username", you could try this: iterate_query = SELECT username FROM accounts > though I have had to comment the following from dovecot.conf: > #iteratedb { > # args = /usr/local/etc/dovecot/dovecot-sql.conf > # driver = sql > #} > because it errors on dovecot start up. Your primary userdb refers to dovecot-sql.lda.conf, not dovecot-sql.conf. What does /usr/local/etc/dovecot/dovecot-sql.lda.conf look like and is there an iterate_query inside this file? Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at knoxvillechristian.org Fri Sep 7 01:50:19 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 06 Sep 2012 18:50:19 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <504928AB.7040702@knoxvillechristian.org> Turn on more debugging with these in your configuration: auth_verbose = yes auth_debug = yes auth_debug_passwords = yes mail_debug = yes verbose_ssl = yes and then check the server logs after trying to login. Also, I'm curious why you masked your PRIVATE ip address and not your public one. Bill On 9/4/2012 12:21 AM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 02:56:37 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 01:56:37 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 Message-ID: <20120906235637.GA11324@daniel.localdomain> Hi, I'm getting segfaults and unexpected disconnects from managesieve server, when the Thunderbird SIEVE extension tries to validate SIEVE scripts agains Pidgeonhole in Dovecot 2.1.8. The extension says: "Server terminated unexpectedly the connection, click on reconnect to try again." It can be reproduced by simply calling CHECKSCRIPT followed by a string: mail01:~# telnet 127.0.0.1 19200 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. "IMPLEMENTATION" "Sieve" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave" "NOTIFY" "mailto" "SASL" "PLAIN" "VERSION" "1.0" OK "Mailbox" AUTHENTICATE "PLAIN" "AGRwYXJ0aGV5QGV4YW1wbGUub3JnAGRwYXJ0aGV5" OK "Logged in." CHECKSCRIPT {6+} abcdef Connection closed by foreign host. The server interrupts the connection and the logs show the following: # dovecot.log Sep 7 01:40:46 dovecot: mailbox: mail: managesieve(dparthey at example.org): Fatal: master: service(managesieve): child 31356 killed with signal 11 (core dumped) # kern.log Sep 7 01:40:46 kernel: [1417105.954609] managesieve[31356]: segfault at 0 ip 00007f1c415c4876 sp 00007fffb3731f88 error 4 in libc-2.11.1.so[7f1c41543000+17a000] Here is the backtrace: mail01:~# gdb /usr/lib/dovecot/managesieve /var/tmp/core.managesieve.31356 Core was generated by `dovecot-mailbox/managesieve'. Program terminated with signal 11, Segmentation fault. #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 (gdb) bt full #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 No symbol table info available. #1 0x000000000040b1c3 in sieve_storage_save_will_activate (ctx=0x257f1a0) at sieve-storage-save.c:328 _data_stack_cur_id = 4 scriptname = 0x25400ae "default.sieve" ret = #2 0x00000000004065b0 in cmd_putscript_finish_parsing (cmd=0x2575cb8) at cmd-putscript.c:206 ehandler = cpflags = sbin = errors = _data_stack_cur_id = 0 script = client = 0x2575c30 ctx = 0x2578180 args = 0x254bc68 ret = #3 0x0000000000406838 in cmd_putscript_continue_script (cmd=0x2575cb8) at cmd-putscript.c:423 all_written = client = 0x2575c30 ctx = 0x2578180 size = 39279968 #4 0x0000000000406caf in client_input_putscript (context=0x2575c30) at cmd-putscript.c:84 cmd = 0x2575cb8 __FUNCTION__ = "client_input_putscript" #5 0x00007f1c41d1d3e6 in io_loop_call_io (io=0x2575fd0) at ioloop.c:379 ioloop = 0x2548680 t_id = 2 #6 0x00007f1c41d1e46f in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 ctx = 0x25489f0 event = 0x2548a60 list = 0x2576020 io = 0x0 tv = {tv_sec = 18, tv_usec = 988469} msecs = ret = i = 0 call = false #7 0x00007f1c41d1d388 in io_loop_run (ioloop=0x2548680) at ioloop.c:398 No locals. #8 0x00007f1c41d09653 in master_service_run (service=0x2548530, callback=0x25400ae) at master-service.c:543 No locals. #9 0x00000000004096ce in main (argc=1, argv=0x2548370) at main.c:308 set_roots = {0x610d60, 0x0} login_set = {auth_socket_path = 0x2540088 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x409810 , failure_callback = 0x409290 } service_flags = storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = xxx imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } From stephan at rename-it.nl Fri Sep 7 03:14:50 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 07 Sep 2012 02:14:50 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 In-Reply-To: <20120906235637.GA11324@daniel.localdomain> References: <20120906235637.GA11324@daniel.localdomain> Message-ID: <50493C7A.1070602@rename-it.nl> On 9/7/2012 1:56 AM, Daniel Parthey wrote: > Hi, > > I'm getting segfaults and unexpected disconnects from managesieve server, > when the Thunderbird SIEVE extension tries to validate SIEVE scripts > agains Pidgeonhole in Dovecot 2.1.8. > > The extension says: > "Server terminated unexpectedly the connection, click on reconnect to try again." > > It can be reproduced by simply calling CHECKSCRIPT followed by a string: This is a known and fixed problem: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/6ceeb6421231 I should make a release soon. Regards, Stephan. From Nico.Weinreich at drefa-msg.de Fri Sep 7 10:30:46 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Fri, 7 Sep 2012 09:30:46 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars Message-ID: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Hi! I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht" => Roundcube shows that messages folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht123" => Roundcube shows error that no messages where found folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung" => Roundcube shows that messages folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung123" => Roundcube shows error saying: "Connection to storage server failed" Looking into /var/log/mail.log shows: Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4478a) [0 x7f065ba8e78a] -> /usr/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f065ba8e7d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f065ba655ef ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a3cb) [0x7f065bd6f3cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a8aa) [0x7f065bd6f8 aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x78641) [0x7f065bd4d641] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_fore ach+0x3c) [0x7f065bd4d7bc] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header+0x29) [0x7f065ba78eb9] -> /usr/lib/dovecot/libdovecot-st orage.so.0(+0x9a0cf) [0x7f065bd6f0cf] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9b160) [0x7f065bd70160] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_storage_search_next_nonblock+0x48) [0x7f065bd706e8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonbl ock+0x27) [0x7f065bd51a07] -> dovecot/imap() [0x41668f] -> dovecot/imap(imap_search_start+0xd8) [0x416ad8] -> dovecot/imap(cmd_sort+0x1e6) [ 0x40e636] -> dovecot/imap(command_exec+0x3d) [0x41157d] -> dovecot/imap() [0x4104fe] -> dovecot/imap(client_handle_input+0x135) [0x410805] - > dovecot/imap(client_input+0x5f) [0x41112f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f065ba9a926] -> /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f065ba9b95f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f065ba9a8c8] -> /usr/li b/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f065ba87053] -> dovecot/imap(main+0x2a4) [0x419354] -> /lib/libc.so.6(__libc_start_ma in+0xfd) [0x7f065b706c8d] -> dovecot/imap() [0x408e79] Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? Regards From ml at cibest.com Fri Sep 7 11:46:42 2012 From: ml at cibest.com (Cibest ML) Date: Fri, 07 Sep 2012 10:46:42 +0200 Subject: [Dovecot] Wrong quota calculation Message-ID: <5049B472.8060407@cibest.com> Hi, We've a strange quota behavior on our dovecot servers. It seems that it's somehow doubling the real used storage (or could be something like twice the inbox size + size of other folders, hard to know for sure). It was initial setup with the maildir++ system. Now it's been configured with dict and mysql table but the result is the same. Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used As example, let's use mailbox at domain.com. If I check in the quota table, I'll find : - size : 697151354 - number of files : 1510 Now if I check the Maildir folder on the filesystem, I'll find : - size : 347960000 - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, not 100% accurate) Another mail box Reported by dovecot : 503446986 1081 Filesystem : 248592000 598 Does anyone has an idea of what could be possibly wrong. We've checked & double-checked everything and nothing seems to be wrong (no dsync used). -- Regards S?bastien From amateo at um.es Fri Sep 7 13:11:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:11:45 +0200 Subject: [Dovecot] best way to activate quota Message-ID: <5049C861.2030904@um.es> Hello, I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). What do you think is the best to activate it? I have think that I could use doveadm commands to calculate an initial quota (on low load periods) and then activate quota, hoping that although quota is not correct, it would be used. Should this work? From robert at schetterer.org Fri Sep 7 13:28:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 07 Sep 2012 12:28:45 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: <5049CC5D.7070005@schetterer.org> Am 07.09.2012 12:11, schrieb Angel L. Mateo: > Hello, > > I'm planning to activate quota control in dovecot, with maildir > quota backend. I have about 70k users in my system directed to 4 backend > servers (with a director to ensure that a user is always directed to the > same server). sounds good, first look > > I have tried to activate it in one of my nodes. The problem is that > load of it has increase a lot, as much as the system was unusable > (maildir is in nfs storage, with indexes in local disks). that might not be the optimal storage solution > > What do you think is the best to activate it? yes ,until its better investigated, cause your systems shouldnt get unusable > > I have think that I could use doveadm commands to calculate an > initial quota (on low load periods) and then activate quota, hoping that > although quota is not correct, it would be used. Should this work? > > with 70 k users , you should be able to hire professional help from timo and/or other dovecot specialists however you ever will have to post your config/logs/general-setup-big-picture if asking for help -- Best Regards MfG Robert Schetterer From amateo at um.es Fri Sep 7 13:41:40 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:41:40 +0200 Subject: [Dovecot] Removing specific entry in user/auth cache In-Reply-To: <1341388872.2689.3.camel@innu> References: <1340865829.25551.64.camel@innu> <42916718-6B7E-4632-8C61-AA8FE64D850E@iki.fi> <1341388872.2689.3.camel@innu> Message-ID: <5049CF64.8090908@um.es> El 04/07/12 10:01, Timo Sirainen escribi?: > On Fri, 2012-06-29 at 05:01 +0300, Timo Sirainen wrote: >> and for v2.1 a bit kludgy way: >> >> doveadm auth [] >> doveadm auth cache flush [] > > Done: http://hg.dovecot.org/dovecot-2.1/rev/007bf0047ab0 > http://hg.dovecot.org/dovecot-2.1/rev/1093c74f54af > Hello, After some time I have updated my system to 2.1.9 which includes this patch but I have doubts it is working. I have changed an attribute for one of my users (his home directory) so I run: root at myotis33:~# doveadm auth cache flush 2 cache entries flushed but, then, when I run "doveadm user " I've got the old information, not the updated one. I had to reload dovecot to get the information correctly reloaded. From hmoreno at gmv.com Fri Sep 7 13:43:19 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Fri, 7 Sep 2012 10:43:19 +0000 Subject: [Dovecot] Quota less mailbox with mdbox Message-ID: Good morning everyone, I have a little doubt about setting a quotaless mailbox with mdbox. Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. The migration is simple and easy, but there are some configuration settings we cannot set to work properly. In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. This is working like a charm. Dovecot.conf: ... namespace { separator = . prefix = inbox = yes type = private } namespace { inbox = no location = prefix = INBOX. separator = . hidden = yes type = private list = no } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } ... plugin/quota = maildir:_backup:ns= ... Dovecot-ldap.conf: ... pass_filter = (&(objectClass=qmailUser)(uid=%u)(accountActive=TRUE)) pass_attrs = uid=user,userPassword=password, mailQuotaSize=quota_rule=*:bytes=%$ user_attrs = mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ user_filter = (&(objectClass=qmailUser)(|(mailAlternateAddress=%u)(mail=%u)(uid=%u))(accountActive=TRUE)) ... When migrated to mdbox and changing the configuration, we don't know how to get it work. We have also tried to follow the instructions from the wiki of dovecot2 without success. Could anyone give us a clue please? If additional information is needed, please let me know. Thank you very much in advanced. Kind regards. ________________________________ [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\logo_gmv.gif] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_blog.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_facebook.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_twitter.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_youtube.gif] P Please consider the environment before printing this e-mail. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From kiwi at oav.net Fri Sep 7 14:30:42 2012 From: kiwi at oav.net (Xavier Beaudouin) Date: Fri, 7 Sep 2012 13:30:42 +0200 Subject: [Dovecot] dsync mirror and sieve ? Message-ID: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Hi there, I am currently doing some extensive test with dsync, but it appear that : dsync -u maildir:/path/to/new/Maildir doesn't seems to sync sieves rules. Is there a good way to sync sieves as well ? :p Regards, Xavier From mhinch at hinchfamily.com Fri Sep 7 14:39:24 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Fri, 07 Sep 2012 06:39:24 -0500 Subject: [Dovecot] dovecot/log not running Message-ID: <5049DCEC.5050008@hinchfamily.com> I'm running 2.1.7 as downloaded/installed by Debian (and then appropriately setup for my config, ...). It almost works. I can send/receive email from thunderbird, squirrelmail, ... However there are two anomalies: 1. dovecot/log is not running. After restarting dovecot (and waiting 30 seconds), I have this in the log files: Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 After 30 seconds, a "ps -ef | grep dovecot" yields: dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil root 25148 20223 0 18:55 ? 00:00:00 dovecot/config dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). A "doveadm log find" dumps this in the log: Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) However, further debugging that I try to do (see next problem) does not result in ANY further input to the log files, even though I have all known logging turned on. So I *think* that logging is not actually running. If I can get that started, then I'll take a crack at the next problem myself ... 2. However, as long as I have your attention, my next problem is that it seems like dovecot/imap-login hangs(?). After a few hours, I have something like: # ps -ef | grep dovecot dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth root 13277 31186 0 17:47 ? 00:00:00 dovecot/config dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil with exactly 100 imap-login processes. I'm the only meaningful user on the machine (unless somebody is imap DoS-ing me), so this does not make much sense. Obviously thunderbird is polling the machine and each time is leaving around a imap-log process. As far as I can tell, thunderbird successfully logs in (i.e., I get mail) though I can't be sure whether these are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? After a while of course, dovecot will not start any new imap-login processes so Thunderbird cannot get any more mail until I restart dovecot. Then everything is OK for another 12 hours or so. Ideas on either problem? -Mark From CMarcus at Media-Brokers.com Fri Sep 7 14:59:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 07 Sep 2012 07:59:44 -0400 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <5049E1B0.2030602@Media-Brokers.com> On 2012-09-07 6:43 AM, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. If you're going to migrate to mdbox, you'd be much better off if you upgraded to the latest 2.1.x... Mdbox is much better supported in the current stable builds. -- Best regards, Charles From trashcan at odo.in-berlin.de Fri Sep 7 15:16:14 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 07 Sep 2012 14:16:14 +0200 Subject: [Dovecot] =?utf-8?q?dsync_mirror_and_sieve_=3F?= In-Reply-To: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> References: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Message-ID: <0ae24afcca736872e3795e930612cb67@mx1.enfer-du-nord.net> Hi -- Xavier Beaudouin wrote: > I am currently doing some extensive test with dsync, but it appear > that : > > dsync -u maildir:/path/to/new/Maildir > > doesn't seems to sync sieves rules. > > Is there a good way to sync sieves as well ? :p I'm using Unison to sync my user's dedicated sievedirs. Regards, Michael From tth at rfa.cz Fri Sep 7 18:34:34 2012 From: tth at rfa.cz (TeTeHacko) Date: Fri, 07 Sep 2012 17:34:34 +0200 Subject: [Dovecot] pop3 migration In-Reply-To: <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> References: <503632C7.2070008@rfa.cz> <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> Message-ID: <504A140A.6070000@rfa.cz> Probably on server side ... the problem was only on big mailboxes which is used by IMAP, so there is no problem and we have already migrated all to dovecot. ;) Thanks for your reply. On 31.8.2012 15:46, Timo Sirainen wrote: > On 23.8.2012, at 16.40, TeTeHacko wrote: > >> I have problems with pop3 migration, the server disconect the dovesync when >> fetching mails after 5 minutes with this error in server log: >> >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6548 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 47522 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6549 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 355289 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6550 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 9638 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6551 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 2188 octets >> xx.xx.xx.xx [28F4] 11:59:05 >>> -ERR Command unrecognized: "" >> xx.xx.xx.xx [28F4] 11:59:05 *** 6551 477693894 00:12:55 ERROR >> xx.xx.xx.xx [28F4] 11:59:05 Disconnected > > Looks like a bug. But in Dovecot POP3 client or the POP3 server? Apparently either Dovecot sends an empty line after RETR or the server thinks Dovecot sends it. Can you get the actual TCP traffic logs? and/or set pop3c_rawlog_dir and see if it's there. > From trashcan at odo.in-berlin.de Fri Sep 7 22:22:48 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 7 Sep 2012 21:22:48 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? Message-ID: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Hi -- Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I just want to be sure that I haven't overseen such a tool/functionality in the documentation. Did I or didn't I? Thanks and regards, Michael From Ralf.Hildebrandt at charite.de Fri Sep 7 22:30:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 7 Sep 2012 21:30:32 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907193032.GD11025@charite.de> * Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. > > Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I think this can be done with post-login scripting, where a script checks if the user is allowed to login. http://wiki.dovecot.org/PostLoginScripting -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From silent_wumpus at yahoo.com Fri Sep 7 22:32:38 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Fri, 7 Sep 2012 12:32:38 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method Message-ID: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From rostetter at mail.utexas.edu Fri Sep 7 23:10:30 2012 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Fri, 07 Sep 2012 15:10:30 -0500 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907151030.44362ukqt9598wqo@mail.ph.utexas.edu> Quoting Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot > (accepted by postfix), temporarily. Now, I would like to know if > there is a way to make a "doveadm kick -A" a permanent disconnect > until one would allow reconnects, again? This should be done > *without* shutting down dovecot or *extensive* re-configuration. A > "doveadm block -u -m 'sorry: running maintenance, please, come > back later'" alike tool/funtionality would be ideal. What I've been doing so far, is I configure dovecot 2.1 with a deny-hosts file (in auth-deny.conf.ext) and then I can append a name to that file to block them, and remove them to allow them back in. Since this file is a simple text file with one username/address per line, it is very easy to manage. No idea if that would work for you, since your usage is for a different reason than mine. -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 23:46:50 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 22:46:50 +0200 Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <1346732123552-37388.post@n4.nabble.com> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> <1346732123552-37388.post@n4.nabble.com> Message-ID: <20120907204650.GA9602@daniel.localdomain> bmalik wrote: > when i login to the server using pop3s, i see in the dovecot log; > > @4000000050457dd9179c653c pop3-login: Info: Login: user=, > method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, > session= > > if i add my mail address into the users-with-forced-encryption file, > i would not reach my account. > > i get an error; > @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script > denied access to user test at byserver.net > whereas i connect to the account via pop3s > if i remove my mail address from the users-with-forced-encryption > there is no problem. > > my postlogin_pop3.sh; > #!/usr/local/bin/bash > if [ "$SECURED" == "secure" ] > then > :# handle secure connections here > else > :# handle insecure connections here > if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption > then > printf "* NO [ALERT] You are not allowed to connect without > encryption\r\n" > exit 0 > fi > fi > exec "$@" > ########### > there is also no problem related to SSL certificate > what can be the issue ? Can you log the value of "$SECURED" to a logfile (using "logger" tool), and log a message in each of the conditional branches of your script, so we can see which branch gets called? Which error message do you get when you manually try to log into POP3S? openssl s_client -connect ip:port USER username PASS password QUIT Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 00:08:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 23:08:33 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1346745602879-37394.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> Message-ID: <20120907210833.GA10535@daniel.localdomain> bmalik wrote: > I upgraded dovecot from 1.1 to 2.1.8 > > pop3/imap works but i have a problem. > > i use pop3 protocol on outlook 2010 for my account. and i set keep my mails > on the server for 5 days or until i delete them. > These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. > if i even delete a mail from outlook the server doesn't delete or old mails > are not deleted. Dovecot does not remove expunged mails from mdbox automatically. You can run doveadm purge via cron to remove unreferenced messages: http://wiki2.dovecot.org/Tools/Doveadm/Purge Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 02:38:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 8 Sep 2012 01:38:44 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5049B472.8060407@cibest.com> References: <5049B472.8060407@cibest.com> Message-ID: <20120907233844.GA12996@daniel.localdomain> Cibest ML wrote: > We've a strange quota behavior on our dovecot servers. > It seems that it's somehow doubling the real used storage (or could > be something like twice the inbox size + size of other folders, hard > to know for sure). > It was initial setup with the maildir++ system. Now it's been > configured with dict and mysql table but the result is the same. > > Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used > As example, let's use mailbox at domain.com. > If I check in the quota table, I'll find : > - size : 697151354 > - number of files : 1510 > > Now if I check the Maildir folder on the filesystem, I'll find : > - size : 347960000 > - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, > not 100% accurate) > > Another mail box > Reported by dovecot : 503446986 1081 > Filesystem : 248592000 598 > > Does anyone has an idea of what could be possibly wrong. We've > checked & double-checked everything and nothing seems to be wrong > (no dsync used). What happens after quota resync? Does the quota change? What happens to the "doveadm quota get" output after LMTP/LDA delivery of a mail with a fixed-size, let's say one mail with 1 MB? How does your setup look like? Mailbox+Director Proxy or only Mailboxes? What is your doveconf -n output of mailbox (and possibly director)? Regards Daniel -- https://plus.google.com/103021802792276734820 From dieterknopf at googlemail.com Sat Sep 8 08:11:34 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 8 Sep 2012 07:11:34 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering Message-ID: Hello, is there any update about the sieve command line tool? I really need some way to refilter my mails without loosings flags and without moving all messages in a different folder. This is possible with procmail via direct access to the file system without the need of getmail. Thanks From robert at schetterer.org Sat Sep 8 09:14:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 08:14:27 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AE243.60207@schetterer.org> Am 08.09.2012 07:11, schrieb Dieter Knopf: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. > > This is possible with procmail via direct access to the file system > without the need > of getmail. > > Thanks > look http://wiki2.dovecot.org/Pigeonhole/Sieve there is some special sieve stuff to filter against external progs i.e. procmail -- Best Regards MfG Robert Schetterer From bmalik at ihlas.net.tr Sat Sep 8 10:25:58 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 00:25:58 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <20120907210833.GA10535@daniel.localdomain> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> Message-ID: <1347089158969-37479.post@n4.nabble.com> But i use Maildir ? is there a solution ? Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i got duplicate mails which i have kept for some days on the server. how can i prevent this problem? -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html Sent from the Dovecot mailing list archive at Nabble.com. From stephan at rename-it.nl Sat Sep 8 10:39:06 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 08 Sep 2012 09:39:06 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AF61A.9090906@rename-it.nl> On 9/8/2012 7:11 AM, Dieter Knopf wrote: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. For Pigeonhole v0.3 it is part of the normal release as the sieve-filter command. I still haven't received much feedback on it though, so make sure your backups are in order before you give it a try and please report back your findings. Regards, Stephan. From robert at schetterer.org Sat Sep 8 16:10:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:10:54 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347089158969-37479.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> Message-ID: <504B43DE.8050008@schetterer.org> Am 08.09.2012 09:25, schrieb bmalik: > But i use Maildir ? > > is there a solution ? > > Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i > got duplicate mails which i have kept for some days on the server. > how can i prevent this problem? > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html > Sent from the Dovecot mailing list archive at Nabble.com. > pop3 feature leave mails on the servers works i.e with thunderbird with a local list in Tb what is allready download for compare so its not only a server feature, so you will allways download all mails new if using a new client without that list, or you changed something on the server ( i.e X-UIDL changed ) and existing list isnt longer build like before see i.e http://kb.mozillazine.org/Popstate.dat however i had some problems with pop3 leave mails on the server with virtual plugin, but that was fixed and works with setup right way -- Best Regards MfG Robert Schetterer From robert at schetterer.org Sat Sep 8 16:12:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:12:53 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: <504B4455.8090600@schetterer.org> Am 08.09.2012 09:39, schrieb Stephan Bosch: > On 9/8/2012 7:11 AM, Dieter Knopf wrote: >> Hello, >> >> is there any update about the sieve command line tool? >> >> I really need some way to refilter my mails without loosings flags and >> without >> moving all messages in a different folder. > > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make > sure your backups are in order before you give it a try and please > report back your findings. > > Regards, > > Stephan. > only for info last time i tried external sieve stuff, it worked with procmail, but i stopped doing something with it -- Best Regards MfG Robert Schetterer From h.reindl at thelounge.net Sat Sep 8 16:16:06 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:16:06 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <504B4516.8040908@thelounge.net> Am 08.09.2012 15:10, schrieb Robert Schetterer: > pop3 feature leave mails on the servers works i.e with thunderbird > with a local list in Tb what is allready download for compare > so its not only a server feature, so you will allways download all mails > new if using a new client without that list, or you changed something > on the server ( i.e X-UIDL changed ) and existing list isnt longer build > like before > > see i.e > > http://kb.mozillazine.org/Popstate.dat > > however i had some problems with pop3 leave mails on the server > with virtual plugin, but that was fixed and works with setup right way and that is why every mail-client has a option "delete messages after x days from server" but users are refusing to spent some minutes of their time looking what options software offers for wondering years later -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From bmalik at ihlas.net.tr Sat Sep 8 16:43:18 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 06:43:18 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <1347111798522-37484.post@n4.nabble.com> i only upgraded dovecot from 1.x to 2.1.x. i didn't change any settings over server. after dovecot service restart, my keep mails on the server, i got again to the outlook. that's my test server but i am going to migrate soon. but i don't want to get duplicate mails. How can i prevent ? Thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html Sent from the Dovecot mailing list archive at Nabble.com. From h.reindl at thelounge.net Sat Sep 8 16:47:41 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:47:41 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347111798522-37484.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B4C7D.108@thelounge.net> Am 08.09.2012 15:43, schrieb bmalik: > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? again: you can NOT POP3 and leave on server depends on the client having a list of message-UUIDs which are already received and it is not uncommon that they are changing due major upgrades on the server DO NOT LEAVE MAILS FOREVER ON THE SERVER this is the only solution and will ever be that is why the client offers "delete after X days from server" smarter clients like thunderbird are defaulting to 14 days not so smart client sleaves their messages forever on the server this is wasting ressources, leads to your troubles and that is why users should learn to look at the options of their sofwtare and try to understand basics who things are working or live with the resulsts if not doing so -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ml at cibest.com Sat Sep 8 18:07:26 2012 From: ml at cibest.com (Cibest ML) Date: Sat, 08 Sep 2012 17:07:26 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120907233844.GA12996@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> Message-ID: <504B5F2E.7070202@cibest.com> On 08/09/2012 01:38, Daniel Parthey wrote: > Cibest ML wrote: >> We've a strange quota behavior on our dovecot servers. >> It seems that it's somehow doubling the real used storage (or could >> be something like twice the inbox size + size of other folders, hard >> to know for sure). >> It was initial setup with the maildir++ system. Now it's been >> configured with dict and mysql table but the result is the same. >> >> Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used >> As example, let's use mailbox at domain.com. >> If I check in the quota table, I'll find : >> - size : 697151354 >> - number of files : 1510 >> >> Now if I check the Maildir folder on the filesystem, I'll find : >> - size : 347960000 >> - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, >> not 100% accurate) >> >> Another mail box >> Reported by dovecot : 503446986 1081 >> Filesystem : 248592000 598 >> >> Does anyone has an idea of what could be possibly wrong. We've >> checked & double-checked everything and nothing seems to be wrong >> (no dsync used). > What happens after quota resync? Does the quota change? > > What happens to the "doveadm quota get" output after LMTP/LDA delivery > of a mail with a fixed-size, let's say one mail with 1 MB? > > How does your setup look like? Mailbox+Director Proxy or only Mailboxes? > > What is your doveconf -n output of mailbox (and possibly director)? > > Regards > Daniel Hi Daniel, Thanks for your answer Here are some tests to show the problem. The test mailbox is initialy empty. All described actions are done in a sequencial way (test mail with an attachement of ~2Mo) - Message delivered by LMTP Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - quota recalc Quota name Type Value Limit User quota STORAGE 5307 524288 User quota MESSAGE 2 - - move to draft + compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move to subfolder "test" in inbox User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move message back to inbox Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - - compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc quota Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - No dovecot director is used at the moment (it's planned). There's one server handling incoming mails (postfix + LMTP), one for IMAP/POP (dovecot) & one for email sending (postfix). dovecot -n output of incoming mails server # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.4.1 Debian 6.0.5 ext4 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_gid = 1500 mail_home = /data/hosting/mail%h/%d/%n mail_location = maildir:/data/hosting/mail%h/%d/%n/Maildir mail_nfs_storage = yes mail_plugins = " quota autocreate" mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Inbox autocreate2 = Trash autocreate3 = Sent autocreate4 = Spam autocreate5 = Drafts autosubscribe = Inbox autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Spam autosubscribe5 = Drafts quota = dict:User quota::noenforcing:proxy::quota quota_rule = *:storage=512M quota_rule2 = Trash:storage=+100M quota_warning = storage=90%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /data/hosting/mail%h/%d/%n/sieve/.dovecot.sieve sieve_before = /etc/dovecot/globalsieve sieve_dir = /data/hosting/mail%h/%d/%n/sieve/ } protocols = " lmtp sieve" quota_full_tempfail = yes service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 10.3.65.71 127.0.0.1 ::1 port = 2055 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } ssl_cert = References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B80A0.9010105@schetterer.org> Am 08.09.2012 15:43, schrieb bmalik: > > > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? > Thanks > dont use pop3, use imap > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- Best Regards MfG Robert Schetterer From haldir.junk at gmail.com Sat Sep 8 23:55:20 2012 From: haldir.junk at gmail.com (Eric Sisolak) Date: Sat, 8 Sep 2012 16:55:20 -0400 Subject: [Dovecot] Vpopmail Dynamic Authentication Module In-Reply-To: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> References: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> Message-ID: On Fri, Aug 31, 2012 at 9:49 AM, Timo Sirainen wrote: > On 24.8.2012, at 16.42, Eric Sisolak wrote: > > > I had to modify the command to build the module, but was able to > > successfully build it: > > gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL > > -I../.. -I../lib -I/home/vpopmail/include -I/home/vpopmail/lib > > userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so > > -L/home/vpopmail/lib/libvpopmail.a > > -L means a path, pointing it to .a file doesn't do anything useful. By > giving the .a file directly probably also fails if libvpopmail.a hasn't > been built with -fPIC. > > > /usr/lib64/dovecot/auth/vpopmail.so: undefined symbol: vclose > > And that's the reason this fails, because libvpopmail isn't included in > vpopmail.so. > > Ah that was it, I was misunderstanding -l and -L. I got it to work with: gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL -I../.. -I../lib -I/home/vpopmail/include -L/home/vpopmail/lib userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so -lvpopmail Thanks! From dieterknopf at googlemail.com Sun Sep 9 06:42:15 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sun, 9 Sep 2012 05:42:15 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: 2012/9/8 Stephan Bosch : > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make sure > your backups are in order before you give it a try and please report back > your findings. Thanks for the info. I'm not really understanding that tool :-( I tried this: sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep But i only get an error: sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' What's the problem? ._REFILTER_ is a standard IMAP subfolder. Thanks From manu at netbsd.org Sun Sep 9 10:17:46 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Sun, 9 Sep 2012 09:17:46 +0200 Subject: [Dovecot] Auth worker max line size In-Reply-To: <19A4E62F-42A0-4FDD-AD68-31C1B26B9E65@iki.fi> Message-ID: <1kq5dlx.15gv4r5j58ya7M%manu@netbsd.org> Timo Sirainen wrote: > Couldn't you change the client to use AUTHENTICATE PLAIN command instead? > The buffer wouldn't be a problem then.. Sorry for the delay, I missed the reply. That is not an option, as the client is not SASL capable. --- src/lib-master/master-auth.h.orig +++ src/lib-master/master-auth.h @@ -13,9 +13,9 @@ /* Authentication client process's cookie size */ #define MASTER_AUTH_COOKIE_SIZE (128/8) /* LOGIN_MAX_INBUF_SIZE should be based on this.*/ -#define MASTER_AUTH_MAX_DATA_SIZE 1024 +#define MASTER_AUTH_MAX_DATA_SIZE 4096 #define MASTER_AUTH_ERRMSG_INTERNAL_FAILURE \ "Internal error occurred. Refer to server log for more information." -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu at netbsd.org From stephan at rename-it.nl Sun Sep 9 11:48:47 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 09 Sep 2012 10:48:47 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: <504AF61A.9090906@rename-it.nl> Message-ID: <504C57EF.4090206@rename-it.nl> On 9/9/2012 5:42 AM, Dieter Knopf wrote: > 2012/9/8 Stephan Bosch : > >> For Pigeonhole v0.3 it is part of the normal release as the sieve-filter >> command. I still haven't received much feedback on it though, so make sure >> your backups are in order before you give it a try and please report back >> your findings. > Thanks for the info. > > I'm not really understanding that tool :-( > > I tried this: > sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep > > But i only get an error: > sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' > > What's the problem? ._REFILTER_ is a standard IMAP subfolder. Mailboxes are not specified in terms of filesystem paths; these are specified in much the same way as for IMAP (albeit in UTF8 where relevant) and doveadm mailbox. So you should try: sieve-filter -u foo at foo.tld main.sieve _REFILTER_ keep The documentation is part of the problem I guess. I've adjusted the man page to specify the mailbox naming explicitly: Regards, Stephan. From roundcube222 at alaadin.org Sun Sep 9 20:51:30 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Sun, 09 Sep 2012 20:51:30 +0300 Subject: [Dovecot] ChgrpNoPerm issue Message-ID: <1b9365b674565d8a794678f3a2858712@Coptics.org> I have dovecot version 2.0.9 I am confused about this article http://wiki2.dovecot.org/Errors/ChgrpNoPerm my mail_location = mbox:~/mymail:INBOX=/var/mail/%u So my inbox is at var/mail/ and my imap folder is /home/user/mymail The problem is /var/mail (Owner is useraccount and the group us mail) , and here comes the problem Dovecot keeps trying to chown the imap folder inside the homedirectory with user:mail account and since And this is the reason of error that appears in my log file. The option mail_access_groups=mail solve the problem.. but I read it is not secure.. With my current configutation, users login with imap to read mail , can they use any the mail_access_groups=mail and read other poeple mail ? does my configutation is a shard mailbox and could be unsecure.. my main quesion , for my current configuration , can i use mail_access_groups=mail safely or can some user use this to read other people mail ? attached my configuration. why didnot have this issue with uw-imapd? and why dovecot try to chown the .imap folder with the mail group ? dovecot -n output is # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <20120909213052.GA11444@daniel.localdomain> Cibest ML wrote: > On 08/09/2012 01:38, Daniel Parthey wrote: > >Cibest ML wrote: > >>We've a strange quota behavior on our dovecot servers. > >>It seems that it's somehow doubling the real used storage (or could > >>be something like twice the inbox size + size of other folders, hard > >>to know for sure). > >>It was initial setup with the maildir++ system. Now it's been > >>configured with dict and mysql table but the result is the same. > Here are some tests to show the problem. The test mailbox is > initialy empty. All described actions are done in a sequencial way > (test mail with an attachement of ~2MB) > > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems ok. > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Strange. There must be some issue with quota recalculation. Maybe there are two instances of the message in the maildir? How large is the actual maildir measured by diskusage (du) command and where are the messages stored in the filesystem? > - move to draft + compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems absolutely correct. Message is copied, deleted and removed from the old directory. > - recalc > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems after compacting the folders there is only one instance of the message. > - move to subfolder "test" in inbox > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > - move message back to inbox > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - The message gets copied and deleted, so there are 2 messages. > - compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - After compaction the "copied+deleted" message gets finally removed. > - recalc quota > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - Very strange. The removed message "reappeared"? Are you using some network filesystem and possibly experience filesystem caching effects? In this case a dovecot director would help. Regards Daniel -- https://plus.google.com/103021802792276734820 From lgb at lgb.hu Mon Sep 10 14:33:08 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Mon, 10 Sep 2012 13:33:08 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" Message-ID: <20120910113307.GG14437@vega.lgb.hu> Hi, I'm trying to use dsync (dovecot version 2.1.9) for IMAP only per-user migration from other IMAP servers to my shiny new one. One worked quite well, but there is a problem with an old IMAP server: dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Looking up IP address dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Connecting to 192.168.10.100:143 dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Disconnected dsync(migtest at office): Error: imapc: Command failed: Disconnected from server dsync(migtest at office): Error: user migtest at office: Initialization failed: Initializing mail storage from mail_location setting failed: imapc: LIST failed: Internal error occurred. Refer to server log for more information. [2012-09-10 11:49:02] dsync(migtest at office): Fatal: User init failed What can cause this? "By hand" test with telnet'ing says: dovecot-test:~$ telnet imap-sunw-old.servers.intra 143 Trying 192.168.10.100... Connected to imap-sunw-old.servers.intra. Escape character is '^]'. * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging Server 6.3-5.02 (built Oct 12 2007; 32bit)) (this is one line of course) Is it possible that the problem about ')' is caused by this line? Or is it tottaly different issue? I just think this, because the the major difference between this and the working server on migration is to have ')' and '(' at the imap "greeting", the working server does not have those, and the error message is about something with ')'. The command was the following (no configuration in dovecot for this, only this command is given at shell level - currently mainly for testing if it works or not); doveadm -D -v -o imapc_features=rfc822.size \ -o imapc_host=192.168.10.100 \ -o imapc_user=migtest at office -o imapc_password=SECRET backup -R -f \ -u migtest at office imapc: This was OK with a dovecot v1 as the "source" (as I've written) but has the problem I've desribed with that "old" sun messaging server as the source. I've also used tcpdump then wireshark's "follow TCP stream" on it: the communication seems to be just that "greeting" line, then my (target) dovecot sends FIN. That "old" server works otherwise since years with various IMAP clients. The log of my server does not contain too much just the db lookups, which works nicely (the same for working and non-working source IMAP servers): Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: prefetch(migtest at office): passdb didn't return userdb entries, trying the next userdb Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): user search: base=cn=mail,dc=office,dc=intra scope=subtree filter=(&(objectClass=mailUser)(uid=migtest at office)) fields=uid Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): result: uid=migtest at office Any help is welcome, - G?bor From ml at cibest.com Mon Sep 10 18:03:39 2012 From: ml at cibest.com (Cibest ML) Date: Mon, 10 Sep 2012 17:03:39 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120909213052.GA11444@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <20120909213052.GA11444@daniel.localdomain> Message-ID: <504E014B.8040309@cibest.com> On 09/09/2012 23:30, Daniel Parthey wrote: > Cibest ML wrote: >> On 08/09/2012 01:38, Daniel Parthey wrote: >>> Cibest ML wrote: >>>> We've a strange quota behavior on our dovecot servers. >>>> It seems that it's somehow doubling the real used storage (or could >>>> be something like twice the inbox size + size of other folders, hard >>>> to know for sure). >>>> It was initial setup with the maildir++ system. Now it's been >>>> configured with dict and mysql table but the result is the same. >> Here are some tests to show the problem. The test mailbox is >> initialy empty. All described actions are done in a sequencial way >> (test mail with an attachement of ~2MB) >> >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems ok. > >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Strange. There must be some issue with quota recalculation. > Maybe there are two instances of the message in the maildir? > How large is the actual maildir measured by diskusage (du) > command and where are the messages stored in the filesystem? > >> - move to draft + compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems absolutely correct. Message is copied, > deleted and removed from the old directory. > >> - recalc >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems after compacting the folders there is > only one instance of the message. > >> - move to subfolder "test" in inbox >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> - move message back to inbox >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > The message gets copied and deleted, > so there are 2 messages. > >> - compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > After compaction the "copied+deleted" > message gets finally removed. > >> - recalc quota >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > Very strange. The removed message "reappeared"? > Are you using some network filesystem > and possibly experience filesystem caching effects? > In this case a dovecot director would help. > > Regards > Daniel The storage is indeed done via NFS. There might be some caching effects but in that case, the behaviour should occur for all the test cases. But it seems only to happen if the mails are stored in INBOX, not in a subfolder of INBOX nor another IMAP folder. We've never been able to reproduce the issue if the email is not exactly in INBOX. We've added some options to check if it's a NFS issue (but it shouldn't be needed as indexes aren't on a NFS filer) : map_disable = yes mail_fsync = always mail_nfs_index = yes It doesn't change anything, same behavior. Here are some test cases to show that there's always 1 mail stored on the filesystem new incoming mail du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota get Quota name Type Value Limit User quota STORAGE 3 524288 User quota MESSAGE 1 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota recacl Quota name Type Value Limit User quota STORAGE 7 524288 User quota MESSAGE 2 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 it really seems to be connected to the recalc operation. Moving to dovecot director would require some work but at the moment, we can't be sure it'd solve the issue (as we only have one IMAP/POP server + one LMTP server, we shouldn't have issues with stickiness of connexions) -- Regards S?bastien From roundcube222 at alaadin.org Mon Sep 10 20:28:37 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 10 Sep 2012 20:28:37 +0300 Subject: [Dovecot] Chown ChgrpNoPerm problem Message-ID: <3d5d646091b201e8545b990fc51c28b8@Coptics.org> Hello, I just have a one simple question. my logs has the following entries Sep 9 02:48:10 dovecot: pop3(r): Error: fchown(/home/r/mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) Sep 9 11:22:30 dovecot: pop3(r): Error: fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) i know that chmod 0600 /var/mail , will solve the problem and i will no longer receive the above errors again But my question is that incase i did not set chmod /var/mail 0600, can i ignore such errors, is these errors harmful ? if this errors keep coming and i ignored them would this cause mbx corruption .. please advise My dovecot conf is disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = hello timo and other dev after a precise reading of plugin deleted to trash it appears to me like things more varariables are declared NULL in the code. it appears to me as being possible to achieve actions even if messages with the same id appears several times in different INBOX when it is the exact order of things -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From ben at indietorrent.org Mon Sep 10 22:19:05 2012 From: ben at indietorrent.org (Ben Johnson) Date: Mon, 10 Sep 2012 15:19:05 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <504E3D29.10102@indietorrent.org> On 9/6/2012 1:56 PM, Ben Johnson wrote: > > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). > >>> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam >> >> That is for version 2.x (note the wiki2)... > > Noted; thank you. > >> I recommend going ahead and upgrading to the latest 2.1.9 and starting >> over. >> > > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle > with this further. > > Thanks again, > > -Ben > So, I setup a development environment that mimics the production environment in question. Must I install Dovecot from source to install Antispam from source? Because I installed Dovecot using a back-ported package ( https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and when I go to install the Antispam plug-in from source, executing the "configure" script yields the following error: # ~/dovecot-antispam-plugin# ./configure [...] checking for dovecot-config in "/usr/local/lib/dovecot"... not found configure: configure: Use --with-dovecot=DIR to provide the path to the dovecot-config file. configure: error: dovecot-config not found I'm assuming that this directory does not exist on my system because I didn't install Dovecot from source. Is this accurate? If so, is my only option to install Dovecot from source and then attempt the same with the Antispam plug-in? Thanks for any pointers, -Ben From silent_wumpus at yahoo.com Tue Sep 11 00:49:12 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Mon, 10 Sep 2012 14:49:12 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <1347313752.51294.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have further read the Migration documentation and someone pointed out to me two good questions: 1 - If I use the modified mb2md.pl -U the program creates a dovecot-uidlist. Or if I can produce this list is there a way I can make dovecot use this list when it creates the dovecot-index file sets the first time it accesses the mailbox? Aka is there a command line option using dovecot or doveadm that creates the indexes based on the dovecot-uidlist that I can script for each mailbox I transfer? 2 - How/where/when do I use "pop3_reuse_xuidl=yes" in the dovecot.conf file for the first time each mailbox is read? Does this option need to be used after the first time (it is not set by default in my new server config). ________________________________ From: Tim E. To: Dovecot List Mailing Sent: Friday, September 7, 2012 3:32 PM Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From jtam.home at gmail.com Tue Sep 11 02:03:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 10 Sep 2012 16:03:46 -0700 (PDT) Subject: [Dovecot] ChgrpNoPerm issue In-Reply-To: References: Message-ID: Robert JR writes: > The problem is /var/mail (Owner is > useraccount and the group us mail) , and here comes the problem > Dovecot > keeps trying to chown the imap folder inside the homedirectory with > user:mail account and since > And this is the reason of error that appears > in my log file. Dovecot is trying to keep the permission of your index caches consistent with that of your mailbox it indexes. The INBOX index cache is kept in your user's mail directory (as per your default settings), although you can change that location. > The option mail_access_groups=mail solve the problem.. > but I read it is not secure.. > With my current configutation, users login > with imap to read mail , can they use > any the mail_access_groups=mail > and read other poeple mail ? Yes, that's the security problem. > does my configutation > is a shard mailbox > and could be unsecure.. Could not parse your question/comment. > why didnot have > this issue with uw-imapd? and why dovecot try to chown the .imap > folder > with the mail group ? uw-imapd was not as picky. The extra consideration for group ownership is so that shared access to mailbox files, and their associated index caches, remain consistent. For example, if you shared a mailbox among your colleagues in group "staff" and the mailbox has group=staff,perm=g+rw, then the index caches will inherit those permissions, and members of group staff can access mailbox and indices alike. [You later write ...] > Sep 9 11:22:30 dovecot: pop3(r): Error: > fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) > failed: Operation not permitted (egid=501(r), group based on > /var/mail/r) > > i know that chmod 0600 /var/mail , will solve the problem > and i will no longer receive the above errors again You also have to make sure that autocreated INBOXs (i.e. a brand new account) does not start out with anything other than 0600. You may have to use dovecot's LDA or twist your LDA's arm to create mailboxes that way. I guess you can also avoid these errors by using memory indices, but you forego the advantages of persistent indices. > But my question is > that incase i did not set chmod /var/mail 0600, can i ignore such > errors, is these errors harmful ? if this errors keep coming and i > ignored them would this cause mbx corruption .. please advise No, you can't ignore these errors. They will break IMAP access to those mailboxes (as you will find out). Joseph Tam From ben at morrow.me.uk Tue Sep 11 04:14:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Tue, 11 Sep 2012 02:14:56 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504E3D29.10102@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> Message-ID: <20120911011456.GA3908@anubis.morrow.me.uk> At 3PM -0400 on 10/09/12 Ben Johnson wrote: > > So, I setup a development environment that mimics the production > environment in question. > > Must I install Dovecot from source to install Antispam from source? > > Because I installed Dovecot using a back-ported package ( > https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and > when I go to install the Antispam plug-in from source, executing the > "configure" script yields the following error: > > # ~/dovecot-antispam-plugin# ./configure > [...] > checking for dovecot-config in "/usr/local/lib/dovecot"... not found > configure: > configure: Use --with-dovecot=DIR to provide the path to the > dovecot-config file. > configure: error: dovecot-config not found > > I'm assuming that this directory does not exist on my system because I > didn't install Dovecot from source. It looks to me like you need to install the dovecot-dev package from that site (if you haven't already), and pass --with-dovecot=/usr/lib/dovecot to configure. Ben From ramon.frontera at uib.es Tue Sep 11 12:41:04 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Tue, 11 Sep 2012 11:41:04 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers Message-ID: Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. We have 2 proxies with director and 4 mail servers. I have configured expire plugin on a single mail server and it works Ok. I don't konw how to configure with 4 mail servers and 2 proxies. My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command Do you have any idea how is the best way to configure expire plugin on this scenario? Thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears From robert at schetterer.org Tue Sep 11 12:58:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 11 Sep 2012 11:58:20 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <504F0B3C.1020105@schetterer.org> Am 11.09.2012 11:41, schrieb Ram?n Frontera: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command > Do you have any idea how is the best way to configure expire plugin on this scenario? > > Thanks in advance. > > -- > Ramon Frontera Gallardo > Universitat de les Illes Balears > > > > > i use one mysql setup for all servers http://wiki2.dovecot.org/Plugins/Expire and doing cron jobs on every server i.e doveadm expunge -A mailbox Trash savedbefore 30d fit the cron times and expunge times and folders to your needs prepare for cron mail messages what got expunged if you have lots off acounts there may be better ways -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Tue Sep 11 15:40:14 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 11 Sep 2012 14:40:14 +0200 Subject: [Dovecot] Is it possible to save only some messages compressed to mdbox? (2.1.6) In-Reply-To: References: <20120820130617.GT24203@ruhr-uni-bochum.de> Message-ID: <20120911124014.GU3002@ruhr-uni-bochum.de> On Thu, Aug 23, 2012 at 11:08:38PM +0300, Timo Sirainen wrote: > > On 20.8.2012, at 16.06, Jost Krieger wrote: > > > So here's the questions: > > > > 1. Is it possible to have compressed/uncompressed mails mixed in an > > mdbox? > > Yes. Compression is detected on a per-mail basis. > > > 2. Will it work via "-o" in lda delivery? > > > > I tried to find out in the source but found it hard to see which options > > are pulled in when. > > -o can override all settings that are handled by the running process itself, so yes that includes compression stuff. Thank you for all your help, but I just can't get it to work. What I have now: For some mailboxes I pt extra parameters in the delivery command (via maildrop): EXTRACOMPRESS="-o debug_log_path=/tmp/lda-debug-log -o mail_debug=yes -o plugin/zlib_save_level=9 -o plugin/zlib_save=bz2" These parameters do arrive in lda: 1. Logging works. 2. If I replace plugin/zlib_save_level=9 with plugin/zlib_save_level=10 I get the correct error message from the zlib plugin. There are no problems in the logs and the mails arrive safely. There is just no compressed material in my mdbox storage files. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From tss at iki.fi Tue Sep 11 16:34:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 16:34:50 +0300 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: On 11.9.2012, at 12.41, Ram?n Frontera wrote: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. From anmeyer at anup.de Tue Sep 11 17:00:38 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:00:38 +0200 Subject: [Dovecot] imap_idle_notify_interval Message-ID: <20120911160038.66c3a387@itx.bitcorner.intern> Hi! I want to change the imap_idle_notify_interval t0 29 minutes and have edited the 20-imap.conf with imap_idle_notify_interval = 29 mins and restarted the server. doveconf -n shows me this: .... protocol imap { imap_idle_notify_interval = 29 mins mail_max_userip_connections = 10 mail_plugins = " quota imap_quota autocreate" } .... doveconf |grep idle shows me the following output: default_idle_kill = 1 mins imap_idle_notify_interval = 2 mins mailbox_idle_check_interval = 30 secs idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 4294967295 secs imap_idle_notify_interval = 29 mins I see two line of imap_idle_notify_interval. What is the right one? Greetings Andreas From tss at iki.fi Tue Sep 11 17:02:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:02:54 +0300 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: <20120911160038.66c3a387@itx.bitcorner.intern> References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: On 11.9.2012, at 17.00, Andreas Meyer wrote: > protocol imap { > imap_idle_notify_interval = 29 mins > mail_max_userip_connections = 10 > mail_plugins = " quota imap_quota autocreate" > } > .... > > doveconf |grep idle shows me the following output: > imap_idle_notify_interval = 2 mins > imap_idle_notify_interval = 29 mins > > I see two line of imap_idle_notify_interval. What is the right one? One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. From anmeyer at anup.de Tue Sep 11 17:06:17 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:06:17 +0200 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: <20120911160617.7bc18f58@itx.bitcorner.intern> Hi! Timo Sirainen wrote: > On 11.9.2012, at 17.00, Andreas Meyer wrote: > > > protocol imap { > > imap_idle_notify_interval = 29 mins > > mail_max_userip_connections = 10 > > mail_plugins = " quota imap_quota autocreate" > > } > > .... > > > > doveconf |grep idle shows me the following output: > > imap_idle_notify_interval = 2 mins > > imap_idle_notify_interval = 29 mins > > > > I see two line of imap_idle_notify_interval. What is the right one? > > One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. So everthing is fine and the imap_idle_notify_interval = 29 mins Thank you! Andreas From tss at iki.fi Tue Sep 11 17:14:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:14:27 +0300 Subject: [Dovecot] ACL MySQL dict: to_user==from_user entries In-Reply-To: <20120829124447.GA16592@sys-241.netcologne.de> References: <20120829124447.GA16592@sys-241.netcologne.de> Message-ID: <217953AF-1EFC-45E3-B5F1-F1BF0938F4FC@iki.fi> On 29.8.2012, at 15.44, Roland Rosenfeld wrote: > But after "SETACL INBOX user at domain lrwstipekxacd" I see a new entry > in my imap_acl_shares table where to_user and from_user both contain > the value "user at domain". > > Does it make sense to run a daily cron job, that removes all entries > with from_user=to_user or may this cause other inconsistencies? Yes. Also fixed in hg: http://hg.dovecot.org/dovecot-2.1/rev/3eaeacc85b4c From tss at iki.fi Tue Sep 11 17:17:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:17:59 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: On 1.9.2012, at 11.10, David Anderson wrote: > I've not found the answer to this question anywhere - please forgive me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. Log errors/warnings to a separate file and watch it. Dovecot's error/warning log typically stays completely empty (except when stopping Dovecot). Anything you find there is a potential error.. > My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Restarting is kind of a kludgy solution, since the only thing it does is to kill all the existing connections and hope that they don't immediately just reconnect back. An equivalent fix is doveadm kick '*' From tss at iki.fi Tue Sep 11 17:24:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:24:46 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50449193.8080101@um.es> References: <50449193.8080101@um.es> Message-ID: On 3.9.2012, at 14.16, Angel L. Mateo wrote: > Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): > > doveadm auth cache flush > 4 cache entries flushed Yep. > So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) Works in my tests. From tss at iki.fi Tue Sep 11 17:31:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:31:16 +0300 Subject: [Dovecot] dovecot proxy ssl-parameter from database? In-Reply-To: <5044B187.6000609@dhag.net> References: <5044B187.6000609@dhag.net> Message-ID: On 3.9.2012, at 16.32, Daniel Funke wrote: > So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) You can't. If disable_plaintext_auth=yes or ssl=required, Dovecot fails the login before any passdb or userdb lookup is done. From tss at iki.fi Tue Sep 11 17:33:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:33:10 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120903141121.GR3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: On 3.9.2012, at 17.11, Jost Krieger wrote: >>> imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) >> >> This looks like it does. Existing emails must not be changed, IMAP >> protocol forbids it (and Dovecot internals don't support it either). >> The only safe way to drop attachments is to assign a new UID for the >> mail, which basically means saving a new message without attachments >> and deleting the old one. The Content-Length header doesn't really >> have anything to do with this error. > > Sorry if I was unclear. This is exactly what the plugin does. It builds > a new message, starting with the old message and leaving out the > unwanted attchment(s). Then it saves the new message, deletes the old > message and finally copies over the flags. > > I had a look at the handling of "Content-Length" in dovecot and had the > impression that a new header was added but the old was not removed (if > formally ok) and on reading the first header wins. But I'm not sure, of > course. Any Content-Length header sent by IMAP client or via LDA input should always get dropped (among some other headers) and replaced with the correct value. So if there's a bug, I doubt it has anything to do with that. But if you can easily reproduce this failure it would be nice to get it fixed.. From ben at indietorrent.org Tue Sep 11 17:40:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 10:40:29 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <20120911011456.GA3908@anubis.morrow.me.uk> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> Message-ID: <504F4D5D.8040207@indietorrent.org> On 9/10/2012 9:14 PM, Ben Morrow wrote: > At 3PM -0400 on 10/09/12 Ben Johnson wrote: >> >> So, I setup a development environment that mimics the production >> environment in question. >> >> Must I install Dovecot from source to install Antispam from source? >> >> Because I installed Dovecot using a back-ported package ( >> https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and >> when I go to install the Antispam plug-in from source, executing the >> "configure" script yields the following error: >> >> # ~/dovecot-antispam-plugin# ./configure >> [...] >> checking for dovecot-config in "/usr/local/lib/dovecot"... not found >> configure: >> configure: Use --with-dovecot=DIR to provide the path to the >> dovecot-config file. >> configure: error: dovecot-config not found >> >> I'm assuming that this directory does not exist on my system because I >> didn't install Dovecot from source. > > It looks to me like you need to install the dovecot-dev package from > that site (if you haven't already), and pass > --with-dovecot=/usr/lib/dovecot to configure. > > Ben > > You nailed it, Ben. Thanks! I downloaded/installed the dovecot-dev package from the same maintainer and was able to build Antispam without issue. The hang-up now seems to be with my configuration. I'm seeing the following error in my email client whenever I attempt to move an email message from, say, Inbox to Junk folder: [SERVERBUG] BUG Unknown internal error. Nothing at all appears in syslog when this happens. I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam , to the letter, I believe. The relevant portions of my configuration are as follows: --------------------- protocol imap { mail_plugins = quota imap_quota antispam } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve antispam_backend = MAILTRAIN antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } --------------------- The permissions and execute bit are set in accordance with other executables in the same directory: # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh I would provide more log content, but the debug options seem to be gone in the forked, >= 2.0 version of this plug-in. Any ideas are very much appreciated. Thanks, -Ben From tss at iki.fi Tue Sep 11 19:02:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:02:33 +0300 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <504647B2.5060804@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> Message-ID: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> On 4.9.2012, at 21.25, Florian Zeitz wrote: > Hello everyone and Timo in particular, > > about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present > in Dovecot. > I had always disliked this a bit, because it replicates a lot of code. > This patch generalizes the HMAC function to take a hash_method struct as > parameter, and changes existing code which uses the "old" HMAC function > to use this new one. > > I'm not really sure this is actually a good idea, but I still felt I > should provide the code in case you would want to merge it upstream. It's otherwise good, but this isn't safe: + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); It assumes that the hmac_init() + hmac_final() is called close to each others. I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) From tss at iki.fi Tue Sep 11 19:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:07:49 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50474093.3010306@um.es> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> Message-ID: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> On 5.9.2012, at 15.07, Angel L. Mateo wrote: > El 05/09/12 13:50, Timo Sirainen escribi?: >> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >> >>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>> >>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>> >>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >> >> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >> > Ok, I'll try this. > > But... is there any way to know how a readdir would be done? What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. From tss at iki.fi Tue Sep 11 19:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:15:37 +0300 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <5048CD0F.9030809@securepoint.de> References: <50471311.2050002@securepoint.de> <5048CD0F.9030809@securepoint.de> Message-ID: <766E466E-3264-42EC-BD35-EEFAE1E2FE17@iki.fi> On 6.9.2012, at 19.19, Dafan Zhai wrote: > I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. > > Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Yes, committed to hg. From tss at iki.fi Tue Sep 11 19:19:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:19:35 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Message-ID: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> On 7.9.2012, at 10.30, Weinreich, Nico wrote: > I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. > > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation This crash was fixed in v2.1.8 I think. > Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. From tss at iki.fi Tue Sep 11 19:20:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:20:02 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: On 7.9.2012, at 13.11, Angel L. Mateo wrote: > Hello, > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > What do you think is the best to activate it? Use dict-file quota instead of Maildir++ quota. From tss at iki.fi Tue Sep 11 19:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:23:22 +0300 Subject: [Dovecot] dovecot/log not running In-Reply-To: <5049DCEC.5050008@hinchfamily.com> References: <5049DCEC.5050008@hinchfamily.com> Message-ID: Something's very wrong. I have no idea what. doveconf -n output might help. On 7.9.2012, at 14.39, Mark Hinch wrote: > I'm running 2.1.7 as downloaded/installed by Debian (and > then appropriately setup for my config, ...). It almost > works. I can send/receive email from thunderbird, squirrelmail, ... > However there are two anomalies: > 1. dovecot/log is not running. After restarting dovecot (and waiting > 30 seconds), I have this in the log files: > > Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 > > After 30 seconds, a "ps -ef | grep dovecot" yields: > > dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login > root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w > mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap > root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot > root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login > dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil > root 25148 20223 0 18:55 ? 00:00:00 dovecot/config > dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth > root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params > mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap > > Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). > A "doveadm log find" dumps this in the log: > Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) > > However, further debugging that I try to do (see next problem) does not result in ANY > further input to the log files, even though I have all known logging turned on. So I *think* > that logging is not actually running. If I can get that started, then I'll take a crack at the > next problem myself ... > > 2. However, as long as I have your attention, my next problem is that it > seems like dovecot/imap-login hangs(?). After a few hours, I have something like: > > # ps -ef | grep dovecot > dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth > root 13277 31186 0 17:47 ? 00:00:00 dovecot/config > dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login > root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot > dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login > root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil > > with exactly 100 imap-login processes. I'm the only meaningful user > on the machine (unless somebody is imap DoS-ing me), so this does not > make much sense. Obviously thunderbird is polling the machine and each > time is leaving around a imap-log process. As far as I can tell, thunderbird > successfully logs in (i.e., I get mail) though I can't be sure whether these > are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? > After a while of course, dovecot will not start any new imap-login processes > so Thunderbird cannot get any more mail until I restart > dovecot. Then everything is OK for another 12 hours or so. > > Ideas on either problem? > > -Mark > From Ralf.Hildebrandt at charite.de Tue Sep 11 19:23:30 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:23:30 +0200 Subject: [Dovecot] FTS indexing Message-ID: <20120911162330.GA25501@charite.de> I'm reacreating all FTS indices on all my users' INBOX folders: mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes doveadm index -u %s -q INBOX In "ps auxwww" I'm seeing this: tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] how can a dovecot/indexer-worker actually access the INBOX of the user "tschuetz" (where "tbudkevy" doesn't have access, due to the different UID)...? Should ~%s/Maildir/lucene-indexes be filled after having run the command sequence for all users? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 19:25:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:25:24 +0300 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <6C88B82F-A2FB-4644-AD0A-CB05A4E6C0ED@iki.fi> On 7.9.2012, at 13.43, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. > > The migration is simple and easy, but there are some configuration settings we cannot set to work properly. > > In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. The only difference should be that you need to switch from Maildir++ quota to dict quota backend. http://wiki2.dovecot.org/Quota/Dict From alec at alec.pl Tue Sep 11 19:29:59 2012 From: alec at alec.pl (A.L.E.C) Date: Tue, 11 Sep 2012 18:29:59 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> Message-ID: <504F6707.1030903@alec.pl> On 09/11/2012 06:19 PM, Timo Sirainen wrote: > The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. This is surprising. Can you provide more info about this issue? -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ben at indietorrent.org Tue Sep 11 19:34:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 12:34:29 -0400 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <504F6815.90407@indietorrent.org> On 9/11/2012 12:23 PM, Timo Sirainen wrote: > Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> >> Not sure on the first issue. On the second issue, out of curiosity, what do you have defined in Thunderbird's Account Settings -> [account in question] -> Server Settings -> Advanced -> Maximum number of server connections to cache? Is this value actually set to 100? -Ben From tss at iki.fi Tue Sep 11 19:38:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:38:17 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911162330.GA25501@charite.de> References: <20120911162330.GA25501@charite.de> Message-ID: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > I'm reacreating all FTS indices on all my users' INBOX folders: > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > how can a dovecot/indexer-worker actually access the INBOX of the > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > different UID)...? indexer-worker runs basically as chroot, but it doesn't bother switching back to root immediately. My guess is that this was a very brief race condition in ps between reading process's UID and the command line. Unless you can easily reproduce this? > Should ~%s/Maildir/lucene-indexes be filled after having run the command > sequence for all users? Only if there was something in the INBOX. Also if users already had previous Lucene indexes it doesn't work, you need to run doveadm fts rescan because dovecot.index remembers which message was last indexed to Lucene. From Ralf.Hildebrandt at charite.de Tue Sep 11 19:54:07 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:54:07 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> Message-ID: <20120911165407.GD25501@charite.de> * Timo Sirainen : > On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > > > I'm reacreating all FTS indices on all my users' INBOX folders: > > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes > > You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? There weren't any. That's why I tried creating them! > > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > > > how can a dovecot/indexer-worker actually access the INBOX of the > > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > > different UID)...? > > indexer-worker runs basically as chroot, but it doesn't bother > switching back to root immediately. My guess is that this was a very > brief race condition in ps between reading process's UID and the > command line. Unless you can easily reproduce this? Can't :) > > Should ~%s/Maildir/lucene-indexes be filled after having run the command > > sequence for all users? > > Only if there was something in the INBOX. My users ALWAYS have something in their inbox :) > Also if users already had previous Lucene indexes it doesn't work, you > need to run doveadm fts rescan because dovecot.index remembers which > message was last indexed to Lucene. OK; I'll try that. Where do I finde the lucene index? I didn't find any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to create the directories. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Tue Sep 11 20:04:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:04:32 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911165407.GD25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> Message-ID: <20120911170432.GE25501@charite.de> * Ralf Hildebrandt : > OK; I'll try that. Where do I finde the lucene index? I didn't find > any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to > create the directories. I fucked up royally: I somehow managed to have fts but not fts_lucene in my list of plugins... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 20:47:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:47:23 +0300 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <20120910113307.GG14437@vega.lgb.hu> References: <20120910113307.GG14437@vega.lgb.hu> Message-ID: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> On 10.9.2012, at 14.33, G?bor L?n?rt wrote: > dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' > * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS > CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO > X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] > imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging > Server 6.3-5.02 (built Oct 12 2007; 32bit)) > > Is it possible that the problem about ')' is caused by this line? I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 From tss at iki.fi Tue Sep 11 20:49:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:49:04 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911170432.GE25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> Message-ID: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> On 11.9.2012, at 20.04, Ralf Hildebrandt wrote: > * Ralf Hildebrandt : > >> OK; I'll try that. Where do I finde the lucene index? I didn't find >> any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to >> create the directories. > > I fucked up royally: I somehow managed to have fts but not fts_lucene > in my list of plugins... It should log errors about that if plugin { fts=lucene } I think? From Ralf.Hildebrandt at charite.de Tue Sep 11 20:51:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:51:45 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> Message-ID: <20120911175145.GJ25501@charite.de> * Timo Sirainen : > > I fucked up royally: I somehow managed to have fts but not fts_lucene > > in my list of plugins... > > It should log errors about that if plugin { fts=lucene } I think? Well, I somehow managed to sprinkle the plugin lines in such a way that no error was triggered. Now I put fts and fts_lucene into mail_plugins and everything works. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From florob at babelmonkeys.de Tue Sep 11 21:07:05 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 11 Sep 2012 20:07:05 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> Message-ID: <504F7DC9.3040408@babelmonkeys.de> Am 11.09.2012 18:02, schrieb Timo Sirainen: > On 4.9.2012, at 21.25, Florian Zeitz wrote: > >> Hello everyone and Timo in particular, >> >> about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present >> in Dovecot. >> I had always disliked this a bit, because it replicates a lot of code. >> This patch generalizes the HMAC function to take a hash_method struct as >> parameter, and changes existing code which uses the "old" HMAC function >> to use this new one. >> >> I'm not really sure this is actually a good idea, but I still felt I >> should provide the code in case you would want to merge it upstream. > > It's otherwise good, but this isn't safe: > > + ctx->ctx = t_malloc(meth->context_size); > + ctx->ctxo = t_malloc(meth->context_size); > > It assumes that the hmac_init() + hmac_final() is called close to each others. I had in fact noticed that. The assumption is currently true for all occurrences, and probably will remain such, but I agree it's better to be safe then sorry. > I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) > Well, either that, or we could use a union of all known context structs there. Possibly plus an i_assert(meth->context_size <= sizeof(union hmac_ctxts)). Or we could use i_malloc() and i_free() under the assumption hmac_init() + hmac_final() calls are always matched. I've a certain preference for the union variant, but it's your call. Regards, Florian From tss at iki.fi Tue Sep 11 21:43:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:43:34 +0300 Subject: [Dovecot] v2.1 memory usage Message-ID: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes? ./configure --with-storages=mdbox sdbox maildir mbox cydir I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :) Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something.. From tss at iki.fi Tue Sep 11 21:48:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:48:08 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <504B5F2E.7070202@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> On 8.9.2012, at 18.07, Cibest ML wrote: > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? From tss at iki.fi Tue Sep 11 21:50:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:50:02 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> On 7.9.2012, at 22.32, Tim E. wrote: > I have a box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Tue Sep 11 23:31:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 23:31:53 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <504F6707.1030903@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> On 11.9.2012, at 19.29, A.L.E.C wrote: > On 09/11/2012 06:19 PM, Timo Sirainen wrote: > >> The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. > > This is surprising. Can you provide more info about this issue? No idea if that's a misconfiguration or what .. or maybe the web browser says it's sending UTF8 but isn't. Anyway, with v2.1.7 it crashes if you send: a SEARCH subject "8bit non-utf8 text" I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation From lgb at lgb.hu Wed Sep 12 00:14:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Tue, 11 Sep 2012 23:14:26 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> References: <20120910113307.GG14437@vega.lgb.hu> <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> Message-ID: <20120911211426.GA15502@vega.lgb.hu> Hi, On Tue, Sep 11, 2012 at 08:47:23PM +0300, Timo Sirainen wrote: [...] > > Is it possible that the problem about ')' is caused by this line? > > I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: I just guessed it was, as the error msg was "server sent invalid input" but the only input from the server according to tcpdump was that line. For sure it's only my oversimplified logic now :) > http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 > http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 Thanks, Timo! With these patches the problem does not show up (honestly, just a quick test after patching/rebuilding, but it seems work now). Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Now I got this, but imho this is totally different issue (mine, not dovcot's) and I will take care about that later. Thanks for your help again! From tss at iki.fi Wed Sep 12 00:29:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 00:29:13 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected > >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected All of these connections had finished handshaking. They simply just seemed to disconnect the connection for no reason. I found one reason for why that could happen, which could explain all of this: http://hg.dovecot.org/dovecot-2.1/rev/24e791bbcf69 The user weakness is remembered until all directors are shut down or all directors have acknowledged the weakness. > >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left The user weakness should have been removed at this point, but looks like there's code missing for it for 1-director-only setup. I'll fix that soon.. > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) These are the weak users causing the trouble. This improves logging: http://hg.dovecot.org/dovecot-2.1/rev/27d3289e1f5c From lgb at lgb.hu Wed Sep 12 01:21:03 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 00:21:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: References: <5049C861.2030904@um.es> Message-ID: <20120911222103.GB15502@vega.lgb.hu> On Tue, Sep 11, 2012 at 07:20:02PM +0300, Timo Sirainen wrote: > On 7.9.2012, at 13.11, Angel L. Mateo wrote: > > > Hello, > > > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > > > What do you think is the best to activate it? > > Use dict-file quota instead of Maildir++ quota. > Sorry to jump into this topic, but can I ask why? Is dict quota is superior over Maildir++ quota in performance in general or it is "only" about the fact, that it's better in the case like above: activating the quota for many users later than the creation of the server itself? The question is interesting for me as well, as I need to implement quota with maildir (on NFS). From tss at iki.fi Wed Sep 12 01:28:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:28:58 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120911222103.GB15502@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> Message-ID: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> On 12.9.2012, at 1.21, G?bor L?n?rt wrote: >>> I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). >>> >>> I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). >>> >>> What do you think is the best to activate it? >> >> Use dict-file quota instead of Maildir++ quota. >> > > Sorry to jump into this topic, but can I ask why? Is dict quota is superior > over Maildir++ quota in performance in general or it is "only" about the > fact, that it's better in the case like above: activating the quota for many > users later than the creation of the server itself? Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. From silent_wumpus at yahoo.com Wed Sep 12 01:39:00 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Tue, 11 Sep 2012 15:39:00 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> Message-ID: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: dsync -v -u user backup user at domain.com All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. I ask this question since the mbox file does not have the associated dovecot indexes. I know Im foobaring/newbying something simple at this point. Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. ________________________________ From: Timo Sirainen To: Tim E. Cc: Dovecot List Mailing Sent: Tuesday, September 11, 2012 2:50 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 7.9.2012, at 22.32, Tim E. wrote: > I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Wed Sep 12 01:48:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:48:53 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Message-ID: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From dmalolepszy at optusnet.com.au Wed Sep 12 08:31:32 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 12 Sep 2012 15:31:32 +1000 Subject: [Dovecot] Strip backslash from usernames Message-ID: <50501E34.2030900@optusnet.com.au> Hi, Is there a way to strip a backslash if it exists at the start of username like \user at example.com within the Dovecot LDA? I am using Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL trickery. Dominic. From alessio at skye.it Wed Sep 12 09:48:19 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 12 Sep 2012 08:48:19 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50503033.9010809@skye.it> Il 12/09/2012 00:28, Timo Sirainen ha scritto: > On 12.9.2012, at 1.21, G?bor L?n?rt wrote: > > >> Sorry to jump into this topic, but can I ask why? Is dict quota is superior >> over Maildir++ quota in performance in general or it is "only" about the >> fact, that it's better in the case like above: activating the quota for many >> users later than the creation of the server itself? > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > But dict quota, for enforce limits, can only be used with dovect as LDA? I'm using Maildir++ with qmail/vpopmail for local delivery and dovecot cannot enforce limits on delivery. With this configuration the only way to update dictionary is "doveadm -A quota rescan". Is correct? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From amateo at um.es Wed Sep 12 10:18:01 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 09:18:01 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> Message-ID: <50503729.1030502@um.es> El 11/09/12 18:07, Timo Sirainen escribi?: > On 5.9.2012, at 15.07, Angel L. Mateo wrote: > >> El 05/09/12 13:50, Timo Sirainen escribi?: >>> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >>> >>>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>>> >>>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>>> >>>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >>> >>> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >>> >> Ok, I'll try this. >> >> But... is there any way to know how a readdir would be done? > > What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. > I meant "when", not how, sorry. But my question is answer with this information. From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:24:28 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:24:28 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:28:51 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:28:51 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0B@s-mcl-mx1.mediacity.drefa.int> Oh sorry, this seems to be an copy&paste failure. "dsr_sub_allocation" was one of the search terms (we run a data warehouse here) ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Timo Sirainen Gesendet: Di 11.09.2012 22:31 An: A.L.E.C Cc: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars [...] I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3837 bytes Desc: not available URL: From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:04:48 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:04:48 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." -searching for a non-UTF-8 term and no mail is found then Roundcube shows correctly a not-found error Until here all is exactly the same, except, that I cannot see the Dovecot crash in /var/log/mail.log anymore. May I assume, that this is now a problem of Roundcube? (But if this is related to Roundcube, why do I still get an connection error (see above)? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Weinreich, Nico Gesendet: Mi 12.09.2012 09:24 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From alec at alec.pl Wed Sep 12 11:16:06 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:16:06 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505044C6.7010504@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: > > -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine > -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:31:08 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:31:08 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl><01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int><01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505044C6.7010504@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0E@s-mcl-mx1.mediacity.drefa.int> Yeah, this fixed it. Will this patch go into source? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von A.L.E.C Gesendet: Mi 12.09.2012 10:16 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. From alec at alec.pl Wed Sep 12 11:37:30 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:37:30 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505049CA.6090202@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: I've found what's really going on here. This is initial search request with utf-8: C: A0003 UID SORT (DATE) UTF-8 ALL OR HEADER SUBJECT {10+} C: L?sung123 HEADER FROM {10+} C: L?sung123 S: * SORT S: A0003 OK Sort completed. Now, because of a bug, fixed in commit I posted before, Roundcube sends another request with search arguments converted to US-ASCII. It should do this only if the first command returns "unsupported charset" response. C: A0004 UID SORT (DATE) US-ASCII ALL OR HEADER SUBJECT {9+} C: L.sung123 HEADER FROM {9+} C: L.sung123 S: A0004 NO Invalid search key The problem is that on that convertion ? is converted to character with code 246 (hex F6), which is from extended ASCII table. This is because in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix this in Roundcube. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ramon.frontera at uib.es Wed Sep 12 11:53:37 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 10:53:37 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure Message-ID: Hello we are running Dovecot 2.1.9. We have 2 proxies with director and 4 mail servers. We have the user information in a LDAP. I try to do: # doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed user mail server ip expire time x.x.x.x 2012-09-12 10:52:17 x.x.x.x 2012-09-12 10:55:53 in the dovecot log i have found: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it What can I do to solve this problem? Thanks in advance. -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears From ml at cibest.com Wed Sep 12 11:58:59 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 12 Sep 2012 10:58:59 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> Message-ID: <50504ED3.5030601@cibest.com> On 11/09/2012 20:48, Timo Sirainen wrote: > On 8.9.2012, at 18.07, Cibest ML wrote: > >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? > Hi, Nothing specific on the namespace side. The only entry is the default one in 10-mail.conf namespace inbox { inbox = yes } I tried to comment it out but it doesn't change anything And no symlinks in the Maildir. Here's the file structure of Maildir for the test mailbox ./dovecot-uidvalidity.504df4e7 ./dovecot-uidvalidity ./.Sent ./.Sent/maildirfolder ./.Sent/new ./.Sent/dovecot.index.log ./.Sent/tmp ./.Sent/dovecot-uidlist ./.Sent/cur ./.Spam ./.Spam/maildirfolder ./.Spam/new ./.Spam/dovecot.index.log ./.Spam/tmp ./.Spam/dovecot-uidlist ./.Spam/cur ./new ./subscriptions ./.Drafts ./.Drafts/maildirfolder ./.Drafts/new ./.Drafts/dovecot.index.log ./.Drafts/tmp ./.Drafts/dovecot-uidlist ./.Drafts/cur ./.Drafts/dovecot-keywords ./dovecot.index.log ./tmp ./dovecot.mailbox.log ./dovecot-uidlist ./.Trash ./.Trash/maildirfolder ./.Trash/new ./.Trash/dovecot.index.log ./.Trash/tmp ./.Trash/dovecot-uidlist ./.Trash/cur ./cur ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa ./dovecot-keywords ./dovecot.index.cache -- Regards S?bastien From alec at alec.pl Wed Sep 12 12:22:27 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 11:22:27 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <505049CA.6090202@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505049CA.6090202@alec.pl> Message-ID: <50505453.4050603@alec.pl> On 09/12/2012 10:37 AM, A.L.E.C wrote: > The problem is that on that convertion ? is converted to character with > code 246 (hex F6), which is from extended ASCII table. This is because > in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix > this in Roundcube. Euh. Of course not PHP issue, the bug was in another place. Fixed. https://github.com/roundcube/roundcubemail/commit/40e1c2d14d11c679d97c00ce831b32d43fad4d32 The fixes are applied to master and release-0.8 branches. ps. sorry for spamming your list, I think it's EOT now. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From amateo at um.es Wed Sep 12 12:54:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 11:54:29 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50505BD5.70907@um.es> El 12/09/12 00:28, Timo Sirainen escribi?: > > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > doveadm quota rescan (not documented)? or doveadm quota recalc (as documented)? Anyway, I'll use dict instead of maildir backend. From mhinch at hinchfamily.com Wed Sep 12 14:13:55 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Wed, 12 Sep 2012 06:13:55 -0500 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <50506E73.30609@hinchfamily.com> Answers for both my helpers - Timo and Ben (Thanks!): Timo: Problem #1 - no dovecot/login doveconf -n output: ============== # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-2-686 i686 Debian wheezy/sid mail_access_groups = mail mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = " imap" ssl_cert = Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> From lgb at lgb.hu Wed Sep 12 14:34:07 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 13:34:07 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <20120912113407.GG22070@vega.lgb.hu> On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: > Activating quota is equally slow, because it needs to scan sizes of all > files. If the size is stored in maildir filename (,S=123) this is pretty > fast, otherwise it's pretty slow because Dovecot stat()s the files, but > afterwards it stores them to dovecot-uidlist file. So the main difference > is that Maildir++ requires rescanning the quota periodically, while dict > quota never rescans unless you run doveadm quota rescan. Ahaa, thanks for your answer. But ... why is rescan needed sometimes with maildir++? If I don't touch maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, not even the LDA, etc), then why is it needed to rescan periodically with maildir? I don't really understand the difference then which needs quota rescan with maildir++ but not with dict quota during the very same usage environment otherwise. Anyway I guess, having dict quota with SQL backend is not so a bad idea: I would be able to check incoming mails (at MXs) if target user will have enough space to hold the mail by using the mail size and quota usage (from SQL what dict quota uses as well) on the MXs using eg postfix policy server ... So I won't generate ugly NDRs later, after accepted mails. Not 100% accurate (quota usage may change while mail hits dovecot) but at least it will stop the majority of those quota related NDRs I have problems with currently. The only thing which keep me away from dict quota with sql backend that it needs much more complex stuff than simply using maildir++ quota, but it's maybe only my lazyness ... thanks again. From ramon.frontera at uib.es Wed Sep 12 16:44:27 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 15:44:27 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <61E579FF-DAB8-4569-BA10-8D599FBD0CA8@uib.es> Hello, I have configured mysql on one server and each mail server use this database for expire plugin. I have configured each mail server as explains on:http://wiki2.dovecot.org/Plugins/Expire But now I have a question; How i configure director server to use the database? I need to configure expire plugin on the director server? thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears El 11/09/2012, a las 15:34, Timo Sirainen escribi?: > On 11.9.2012, at 12.41, Ram?n Frontera wrote: > >> Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. >> We have 2 proxies with director and 4 mail servers. >> >> I have configured expire plugin on a single mail server and it works Ok. >> I don't konw how to configure with 4 mail servers and 2 proxies. >> My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > > The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. > From zinovik.igor at gmail.com Wed Sep 12 16:52:11 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Wed, 12 Sep 2012 17:52:11 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes Message-ID: Hello, list. I'm asking question not directly connected with dovecot. I'm planning a migration from cyrus-imapd to dovecot and i'm using cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes of users, but i'm wondering why size of mailbox in Maildir++ is so much bigger than mailbox in cyrus format after conversion: linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik /srv/vmail/petrsu.ru/z/zinovik/Maildir 238M /mnt/imap/z/user/zinovik 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir I was planning to implement quota for mailboxes about 1 gigabyte, but after conversion I would not be able to receive messages to my own box, because i'm overquota. I think the only way would be to set quota up to 15 GB since some of users have mailbox with 12GB after conversion, but I have no such disk capacity to allow more than 1500 users have 15GB quota. Would it be practical solution to compress all files after conversion and enable zlib support in dovecot? From amateo at um.es Wed Sep 12 17:18:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 16:18:47 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120912113407.GG22070@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> <20120912113407.GG22070@vega.lgb.hu> Message-ID: <505099C7.40709@um.es> El 12/09/12 13:34, G?bor L?n?rt escribi?: > On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: >> Activating quota is equally slow, because it needs to scan sizes of all >> files. If the size is stored in maildir filename (,S=123) this is pretty >> fast, otherwise it's pretty slow because Dovecot stat()s the files, but >> afterwards it stores them to dovecot-uidlist file. So the main difference >> is that Maildir++ requires rescanning the quota periodically, while dict >> quota never rescans unless you run doveadm quota rescan. > > Ahaa, thanks for your answer. > > But ... why is rescan needed sometimes with maildir++? If I don't touch > maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) > just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, > not even the LDA, etc), then why is it needed to rescan periodically with > maildir? I don't really understand the difference then which needs quota > rescan with maildir++ but not with dict quota during the very same usage > environment otherwise. > > Anyway I guess, having dict quota with SQL backend is not so a bad idea: I > would be able to check incoming mails (at MXs) if target user will have > enough space to hold the mail by using the mail size and quota usage (from > SQL what dict quota uses as well) on the MXs using eg postfix policy server > ... So I won't generate ugly NDRs later, after accepted mails. Not 100% > accurate (quota usage may change while mail hits dovecot) but at least it > will stop the majority of those quota related NDRs I have problems with > currently. > > The only thing which keep me away from dict quota with sql backend that it > needs much more complex stuff than simply using maildir++ quota, but it's > maybe only my lazyness ... > You could also use dict quota stored in a file in the user's home directory as maildir++ quota does. This is the way I'm going to use it, not with a sql backend. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From lgb at lgb.hu Wed Sep 12 17:40:00 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:40:00 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb Message-ID: <20120912144000.GB1700@vega.lgb.hu> Hi, I'm about getting to know dovecot in details, I am about the command "doveadm user". The man page shows this example: userdb: jane uid : 8001 gid : 8001 home : /home/jane mail : sdbox:~/sdbox plugins : sieve quota_rule: *:storage=150M It seems to be nice, however I never got this only: ~# doveadm user lgb at office userdb: lgb at office That's all. The dovecot test configuration works otherwise via IMAP/POP3/LMTP nicely. What I guess that the lack of extra information (which would be needed by some scripts of mine, eg for getting user's home easily from shell scripts, as it's hash based, etc) is caused because I don't pass home (and/or other settings) back from userdb and/or passdb. However I do have these in global scope: mail_home = /mailstorage/%Ld/%Ln (yes, this is not hashed at all at the momement but it will be soon) mail_location = maildir:~/Maildir mail_uid = vmail mail_gid = vmail (... and also some global quota stuffs configured in plugin section) Now, I am a bit confused: even with passdb/userdb does not return with user home or GID/UID, dovecot knows these in case of pop3/imap/lmtp access, since these information can be "composed" from those settings. But then, wouldn't it possible so "doveadm user" shows those as well, like lmtp/pop3/imap can resolve those too? Honestly, it's a bit redundant to put needed information into user and pass attrs in every dbs (I have some) when dovecot knows those otherwise ... If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb query tool nothing more), is there any tool which works in this config, displaying extra information (at least user's home), or should I create some script which just repeats the functionality of dovecot's configured mail_home resolution? Thanks. From lgb at lgb.hu Wed Sep 12 17:48:52 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:48:52 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb In-Reply-To: <20120912144000.GB1700@vega.lgb.hu> References: <20120912144000.GB1700@vega.lgb.hu> Message-ID: <20120912144852.GA2372@vega.lgb.hu> On Wed, Sep 12, 2012 at 04:40:00PM +0200, G?bor L?n?rt wrote: > If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb > query tool nothing more), is there any tool which works in this config, > displaying extra information (at least user's home), or should I create some > script which just repeats the functionality of dovecot's configured > mail_home resolution? Arghhh, never mind, it seems switch -m does this :) However this is not documented in the man page, I just discovered because "doveadm user" (without user name) lists that, and I tried out what it does ... From kgc at corp.sonic.net Wed Sep 12 19:55:41 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 12 Sep 2012 09:55:41 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> Message-ID: <5050BE8D.2090002@corp.sonic.net> Timo, I'm glad to hear that you found a plausible cause and have got a patch in to fix it. -K From silent_wumpus at yahoo.com Wed Sep 12 20:51:35 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 10:51:35 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> Message-ID: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From silent_wumpus at yahoo.com Wed Sep 12 22:14:24 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 12:14:24 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> Message-ID: <1347477264.9603.YahooMailNeo@web121506.mail.ne1.yahoo.com> Added these lines to /etc/dovecot.conf namespace { separator = / inbox = yes } Ran the command and it works. However it doesnt keep the UID as the thunderbird client redownloads all the POP3 mails. ________________________________ From: Tim E. To: Dovecot Mailing List Sent: Wednesday, September 12, 2012 1:51 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:38:45 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:38:45 +0200 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <50501E34.2030900@optusnet.com.au> References: <50501E34.2030900@optusnet.com.au> Message-ID: <20120913003845.GA10550@daniel.localdomain> Dominic Malolepszy wrote: > Is there a way to strip a backslash if it exists at the start of > username like \user at example.com within the Dovecot LDA? I am using > Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL > trickery. Address rewriting should be done in the MTA. I see no point why this should be done in the LDA? Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:47:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:47:04 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names Message-ID: <20120913004704.GA10738@daniel.localdomain> Hi, the Drafts special folder is called "Entw?rfe" in German locale. How should the special_use entry in the config file look like? Would you recommended to use the encoded folder name... mailbox "Entw&APw-rfe" { special_use = \Drafts } ...or rather the utf-8 Encoding in the dovecot config file? mailbox "Entw?rfe" { special_use = \Drafts } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:54:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:54:57 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure In-Reply-To: References: Message-ID: <20120913005457.GA10928@daniel.localdomain> Ram?n Frontera wrote: > we are running Dovecot 2.1.9. > We have 2 proxies with director and 4 mail servers. > We have the user information in a LDAP. > > I try to do: > # doveadm director map > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > user mail server ip expire time > x.x.x.x 2012-09-12 10:52:17 > x.x.x.x 2012-09-12 10:55:53 > > in the dovecot log i have found: > dovecot: auth: Error: Trying to iterate users, but userdbs don't support it Please see http://wiki2.dovecot.org/Tools/Doveadm/Mailbox Make sure that the iterate_attrs and iterate_filter settings in /etc/dovecot/dovecot-ldap.conf.ext match your LDAP schema and the user is allowed to iterate the LDAP tree. Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Thu Sep 13 09:34:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 13 Sep 2012 08:34:36 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <50517E7C.5070200@schetterer.org> Am 13.09.2012 02:47, schrieb Daniel Parthey: > Hi, > > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } > > Regards > Daniel > which german client does mailbox "Entw&APw-rfe" and is able to use special_use i ve testet k9 mail 4.2 german, no need for translate \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i think this or equal should be the right way for all clients i ve done this for Archive ( to goal Tb Layout, which hasnt special_use yet), but still untested, so try test something for like this for Drafts too, but for now i dont know a client wich is ready and out for special use with german language ( the new outlook will be, but its not out in german yet , as my latest look on it, the beta is only in englisch or spanish ) mailbox Archive { special_use = \Archive auto=no } mailbox "Archives" { special_use = \Archive auto=subscribe } login telnet . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE SEARCH=FUZZY COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in . LIST (SPECIAL-USE) "" "*" * LIST (\Trash) "/" "Trash" * LIST (\Junk) "/" "Junk" * LIST (\Sent) "/" "Sent" * LIST (\Drafts) "/" "Drafts" * LIST (\Archive) "/" "Archives" * LIST (\Unmarked) "/" "INBOX" -- Best Regards MfG Robert Schetterer From zinovik.igor at gmail.com Thu Sep 13 15:55:00 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Thu, 13 Sep 2012 16:55:00 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes In-Reply-To: References: Message-ID: Completly PEBKAC problem. I was not cleaning Maildir directory after several conversions. 2012/9/12 Igor Zinovik : > Hello, list. > > I'm asking question not directly connected with dovecot. > > I'm planning a migration from cyrus-imapd to dovecot and i'm using > cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes > of users, but i'm wondering why size of mailbox in Maildir++ is so > much bigger than mailbox in cyrus format after conversion: > linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik > /srv/vmail/petrsu.ru/z/zinovik/Maildir > 238M /mnt/imap/z/user/zinovik > 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir > > I was planning to implement quota for mailboxes about 1 gigabyte, > but after conversion I would not be able to receive messages to my own > box, because i'm overquota. > > I think the only way would be to set quota up to 15 GB since some of users > have mailbox with 12GB after conversion, but I have no such disk capacity > to allow more than 1500 users have 15GB quota. > > Would it be practical solution to compress all files after conversion and enable > zlib support in dovecot? From ben at indietorrent.org Thu Sep 13 20:25:18 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 13 Sep 2012 13:25:18 -0400 Subject: [Dovecot] antispam_plugin prevents IMAP login (error 3) [Dovecot 2.0.19] In-Reply-To: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> References: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> Message-ID: <505216FE.20807@indietorrent.org> > On 17.08.2012 12:06, dof at projektfabrik.com wrote: >> Hi everybody, >> >> trying to get the Dovecot antispam_plugin to work and I must be doing >> something wrong, because as soon as it is enabled with a certain >> backend, imap logins do not work anymore (the session is immediately >> closed after a successful login). Interestingly, pipe and spool2dir >> are working (that is, the session won't be closed), dspam-exec and >> crm114-exec are not. If this happens, mail_debug output is as >> follows: >> >> [...] > > Hi again, > > how embarrassing. I should use the correct name for the backend type. > :\ So everything works as expected, sorry for spamming the mailing list. > > > Best regards, > Florian Florian, I am having a very similar (or the same) issue. Ubuntu 12.04.1 LTS with Dovecot 2.0.19. Do you mind sharing the details of your configuration, and what, exactly, you changed with respect to the back-end to resolve the issue? a.) Which back-end are you using? b.) What are your "plugin {}" configuration directives? I'm attempting to use mailtrain, per http://wiki2.dovecot.org/Plugins/Antispam . My configuration directives are: plugin { antispam_backend = mailtrain antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } Thanks for any help! -Ben From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 22:23:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 21:23:08 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <50517E7C.5070200@schetterer.org> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> Message-ID: <20120913192308.GA9400@daniel.localdomain> Robert Schetterer wrote: > > the Drafts special folder is called "Entw?rfe" in German locale. > > How should the special_use entry in the config file look like? > > > > Would you recommended to use the encoded folder name... > > > > mailbox "Entw&APw-rfe" { > > special_use = \Drafts > > } > > > > ...or rather the utf-8 Encoding in the dovecot config file? > > > > mailbox "Entw?rfe" { > > special_use = \Drafts > > } > > which german client does mailbox "Entw&APw-rfe" and is able to use special_use? > > i ve tested k9 mail 4.2 german, no need for translate > \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i > think this or equal should be the right way > for all clients I'm not sure which client created it and if any of my clients does support special_use, but I found the following mailbox on our mdbox filesystem: # ls -ld mail/mailboxes/Entw\&APw-rfe drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe Seems like k9mail or Android Mail must have created it in May 2012, the folder is not subscribed and contains only a single mail. The rest is in "Drafts" mailbox (where it belongs). Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Fri Sep 14 08:33:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 14 Sep 2012 07:33:25 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913192308.GA9400@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> <20120913192308.GA9400@daniel.localdomain> Message-ID: <5052C1A5.40409@schetterer.org> Am 13.09.2012 21:23, schrieb Daniel Parthey: > Robert Schetterer wrote: >>> the Drafts special folder is called "Entw?rfe" in German locale. >>> How should the special_use entry in the config file look like? >>> >>> Would you recommended to use the encoded folder name... >>> >>> mailbox "Entw&APw-rfe" { >>> special_use = \Drafts >>> } >>> >>> ...or rather the utf-8 Encoding in the dovecot config file? >>> >>> mailbox "Entw?rfe" { >>> special_use = \Drafts >>> } >> >> which german client does mailbox "Entw&APw-rfe" and is able to use special_use? >> >> i ve tested k9 mail 4.2 german, no need for translate >> \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i >> think this or equal should be the right way >> for all clients > > I'm not sure which client created it and if any of my clients > does support special_use, but I found the following mailbox on > our mdbox filesystem: > > # ls -ld mail/mailboxes/Entw\&APw-rfe > drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe > > Seems like k9mail or Android Mail must have created it in May 2012, > the folder is not subscribed and contains only a single mail. The > rest is in "Drafts" mailbox (where it belongs). > > Regards > Daniel > I would recommand stay cool until you can verify german clients using imap special_use i dont think it makes sense react now, Timo said he might will create some wiki page in relate clients matrix with special use -- Best Regards MfG Robert Schetterer From gcharot at evenium.com Fri Sep 14 11:59:57 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 10:59:57 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail Message-ID: <5052F20D.7050706@evenium.com> Hello all, I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot of lock and "sieve_execute_bytecode error" in my dovecot.log. $ cat /etc/redhat-release CentOS release 5.8 (Final) $ dovecot --version 1.0.13 The sieve scripts are generated by squirrelmail mail and are mostly mail filters and vacations notices. $ yum info squirrelmail.noarch Installed Packages Name : squirrelmail Arch : noarch Version : 1.4.8 Release : 5.el5.centos.13 When I look at my dovecot.log I have a lot of sieve error : deliver(foo): Sep 11 15:08:04 Info: msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: save failed to EvSUPPORT: Timeout while wait ing for lock deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: Generic Error deliver(foo): Sep 11 15:08:04 Error: sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed The server hosts around 50 mailboxes, some people does have sieve errors some doesn't. Is there any way to get more details about the runtime error ? How can I debug this issue ? Thanks a lot in advance, Greg My config below : #dovecot -n # 1.0.13: /etc/dovecot.conf log_path: /var/log/dovecot.log ssl_ca_file: /etc/pki/roch_crt/roch_evenium_com.ca-bundle ssl_cert_file: /etc/pki/roch_crt/roch_evenium_com.crt ssl_key_file: /etc/pki/roch_crt/private/roch-private.key ssl_cipher_list: ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+SSLv3 login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_process_per_connection: no login_process_size: 256 login_processes_count: 10 login_max_connections: 32 mail_privileged_group: mail dotlock_use_excl: yes fsync_disable: yes maildir_copy_with_hardlinks: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls,oe-ns-eoh auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 From amateo at um.es Fri Sep 14 12:13:03 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 14 Sep 2012 11:13:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049CC5D.7070005@schetterer.org> References: <5049C861.2030904@um.es> <5049CC5D.7070005@schetterer.org> Message-ID: <5052F51F.3040605@um.es> El 07/09/12 12:28, Robert Schetterer escribi?: >> >> I have tried to activate it in one of my nodes. The problem is that >> load of it has increase a lot, as much as the system was unusable >> (maildir is in nfs storage, with indexes in local disks). > > that might not be the optimal storage solution > Why do you say it might be not optimal? I think that nfs storage is a very common scenario. From carsten.delellis at delellis.net Fri Sep 14 12:27:23 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Fri, 14 Sep 2012 11:27:23 +0200 Subject: [Dovecot] Couldn't drop privileges: User is missing UID (see mail_uid setting) Message-ID: Hi all I hope anyone could help me out with my problem regarding the user_id settings. I am running my mail servers just for fun and I am not a professional mail admin. After I am using postfix and dovecot for my mail servers with a mysql backend for the last 2 years I have tried to set up an installation with a ldap backend. I followed the how-to from the ?Postfix Book? and now I have always the following error message: Couldn't drop privileges: User is missing UID (see mail_uid setting) This error message occurs whenever I try to connect via a mail client. System emails were delivered the way I expected. So I assume postfix is working fine. The ldap search I use for user authentication is as follows: uris = ldap://myserver.com dn = cn=admin, dc=mydomain, dc=com dnpass = secure tls = no base = ou=mailuser,dc=mydomain,dc=com ldap_version = 3 user_attrs = pfMailMessageStore=home user_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) pass_attrs = pfClearPassword=password,pfMailMessageStore=userdb_home pass_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) default_pass_scheme = PLAIN user_global_uid = 5000 user_global_gid = 5000 I am using dovecot 2.1.9, postfix 2.8.5 and openldap 2.4.23 I would appreciate any help or tipps. ~ Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From stephan at rename-it.nl Fri Sep 14 12:48:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 11:48:21 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052F20D.7050706@evenium.com> References: <5052F20D.7050706@evenium.com> Message-ID: <5052FD65.8020104@rename-it.nl> Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: > Hello all, > > I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot > of lock and "sieve_execute_bytecode error" in my dovecot.log. > > $ cat /etc/redhat-release > CentOS release 5.8 (Final) > > $ dovecot --version > 1.0.13 Very, very, very, very old. > The sieve scripts are generated by squirrelmail mail and are mostly > mail filters and vacations notices. > > $ yum info squirrelmail.noarch > Installed Packages > Name : squirrelmail > Arch : noarch > Version : 1.4.8 > Release : 5.el5.centos.13 > > When I look at my dovecot.log I have a lot of sieve error : > > deliver(foo): Sep 11 15:08:04 Info: > msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: > save failed to EvSUPPORT: Timeout while wait > ing for lock There's the actual error. This has very little to do with Sieve itself. Looks like some mbox locking issue, but I don't really know. You could try enabling mail_debug to get more info. > deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: > Generic Error > deliver(foo): Sep 11 15:08:04 Error: > sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed Yuck, I was hoping that people would finally have stopped using cmusieve. Regards, Stephan. From Jost.Krieger+dovecot at rub.de Fri Sep 14 13:47:59 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 14 Sep 2012 12:47:59 +0200 Subject: [Dovecot] Possible mbox corruption In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: <20120914104758.GU30629@ruhr-uni-bochum.de> On Tue 11 Sep 2012 05:33:10 PM GMT, Timo Sirainen wrote: > Any Content-Length header sent by IMAP client or via LDA input should > always get dropped (among some other headers) and replaced with the > correct value. So if there's a bug, I doubt it has anything to do with > that. But if you can easily reproduce this failure it would be nice to > get it fixed.. You're right, of course. And I cannot reproduce the problem. The user that could reproduce it has somehow given up. I'v experimented and read the source, and there is no way a "Content-Length" could be saved by dovecot :-) But ... This is on a system where delivery is not (yet) via dovecot, but directly by qmail. Two possibilities come to mind: 1. We have a locking problem that only surfaces when qmail and dovecot try to write at the same time to the end of the mbox. Most people (and clients) hardly ever do an APPEND to the INBOX. 2. Something qmail writes to the INBOX makes dovecot hiccup (like a wrong Content-Length header :-;). Which would anyone think more likely? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From gcharot at evenium.com Fri Sep 14 14:03:25 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 13:03:25 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052FD65.8020104@rename-it.nl> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> Message-ID: <50530EFD.8080907@evenium.com> Thanks for the quick reply Stephan ! Is it possible to upgrade from 1.0.13 (which is the official CentOS release) to the atrpms version 2.1.1-2 available at http://packages.atrpms.net/dist/el5/dovecot/ Do I need a migration plan ? Thanks ! Greg Le 14/09/2012 11:48, Stephan Bosch a ?crit : > Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: >> Hello all, >> >> I'm using dovecot on CentOS 5.8 (official CentOS package) and have >> lot of lock and "sieve_execute_bytecode error" in my dovecot.log. >> >> $ cat /etc/redhat-release >> CentOS release 5.8 (Final) >> >> $ dovecot --version >> 1.0.13 > > Very, very, very, very old. > >> The sieve scripts are generated by squirrelmail mail and are mostly >> mail filters and vacations notices. >> >> $ yum info squirrelmail.noarch >> Installed Packages >> Name : squirrelmail >> Arch : noarch >> Version : 1.4.8 >> Release : 5.el5.centos.13 >> >> When I look at my dovecot.log I have a lot of sieve error : >> >> deliver(foo): Sep 11 15:08:04 Info: >> msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: >> save failed to EvSUPPORT: Timeout while wait >> ing for lock > > There's the actual error. This has very little to do with Sieve > itself. Looks like some mbox locking issue, but I don't really know. > You could try enabling mail_debug to get more info. > >> deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: >> Generic Error >> deliver(foo): Sep 11 15:08:04 Error: >> sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed > > Yuck, I was hoping that people would finally have stopped using cmusieve. > > Regards, > > Stephan. From stephan at rename-it.nl Fri Sep 14 14:37:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 13:37:07 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <50530EFD.8080907@evenium.com> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> <50530EFD.8080907@evenium.com> Message-ID: <505316E3.5010802@rename-it.nl> Op 9/14/2012 1:03 PM, Gregory Charot (EVENIUM) schreef: > Thanks for the quick reply Stephan ! > > Is it possible to upgrade from 1.0.13 (which is the official CentOS > release) to the atrpms version 2.1.1-2 available at > > http://packages.atrpms.net/dist/el5/dovecot/ > > Do I need a migration plan ? http://wiki2.dovecot.org/Upgrading?action=show&redirect=UpgradingDovecot http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Migration Regards, Stephan. From roundcube222 at alaadin.org Fri Sep 14 13:37:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 14 Sep 2012 13:37:31 +0300 Subject: [Dovecot] POP3S PORT Message-ID: Hello I am using dovecot with POP3S and IMAPS ports, When client connect to POP3S port (995) , i can see in dovecot log file that the protocol is TLSv1 So how come the protocol is TLSv1 and not SSL ? Does the user connect t port 995 with SSL protocol then encrypt again inside the SSL session to make The TLSv1 protocol ? Or does the client use TLSv1 Protocol immediately when connect to port 995 without Using SSL protocol ? Please Help. Robert JR From mailinglist at darac.org.uk Fri Sep 14 14:49:15 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 14 Sep 2012 12:49:15 +0100 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <20120914114915.GA23431@darac.org.uk> On Fri, Sep 14, 2012 at 01:37:31PM +0300, Robert JR wrote: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? First of all, read http://wiki2.dovecot.org/SSL and see if that answers your question. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 14 15:06:32 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 14 Sep 2012 14:06:32 +0200 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <50531DC8.207@thelounge.net> Am 14.09.2012 12:37, schrieb Robert JR: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? because it is simply the same read about what TLS is -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From damoxc at gmail.com Fri Sep 14 17:46:39 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 15:46:39 +0100 Subject: [Dovecot] key -> object mailstore Message-ID: Hi, I was wondering what would be entailed in modifying dovecot 2.2 to support storing mail in an object store. I've seen a few mails dotted around in the ML history about supporting such a thing and seen it's basically dependant upon some changes in lib-storage to support writing messages without locking. Is this still the case? Regards, Damien From tss at iki.fi Fri Sep 14 17:59:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 14 Sep 2012 17:59:17 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: Message-ID: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> On 14.9.2012, at 17.46, Damien Churchill wrote: > I was wondering what would be entailed in modifying dovecot 2.2 to > support storing mail in an object store. I've seen a few mails dotted > around in the ML history about supporting such a thing and seen it's > basically dependant upon some changes in lib-storage to support > writing messages without locking. Is this still the case? I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? The generic idea is: - only one server accesses one user simultaneously - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync - support high latency: asynchronous reads/writes. prefetch mail bodies. From damoxc at gmail.com Fri Sep 14 18:16:09 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 16:16:09 +0100 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: On 14 September 2012 15:59, Timo Sirainen wrote: > On 14.9.2012, at 17.46, Damien Churchill wrote: > >> I was wondering what would be entailed in modifying dovecot 2.2 to >> support storing mail in an object store. I've seen a few mails dotted >> around in the ML history about supporting such a thing and seen it's >> basically dependant upon some changes in lib-storage to support >> writing messages without locking. Is this still the case? > > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? > That's good to hear :) I've been evaluating RADOS as an object store, which is similar to S3. Although any distributed storage would be great. I'd be more than happy to help code it! > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. > I'm assuming that the director would be used in order to distribute connections to the same server, so it's only within a local instance of dovecot you'd need to be aware of what currently has a connection open for that user? How are you planning on handling the situation where say node X dies and hasn't uploaded the latest index file? Would that result in missing messages from the mailbox when accessed by another node, or is the local index intended to be more of a write-through cache? From ncjeffgus at zimage.com Sat Sep 15 06:39:36 2012 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Fri, 14 Sep 2012 20:39:36 -0700 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <1347680376.5717.61.camel@maclinux> On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it > for v2.2. Do you want to help coding it? :) Which storage would you > want to use? > > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and > accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split > brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. With this system, would the read/write ultimately go to a normal OS file function? If it is a file function, could this be used with a system like glusterfs, ceph, etc? The other option would be to write it against a object store client library and bypass the normal file functions. ...Jeff From patrickdk at patrickdk.com Sat Sep 15 19:50:47 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sat, 15 Sep 2012 12:50:47 -0400 Subject: [Dovecot] v2.1 memory usage In-Reply-To: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> References: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> Message-ID: <20120915125047.Horde.iNAEcZLnE6FQVLHnJ7HDDyA@mail.patrickdk.com> Only have 650 imap clients this weekend, have to give it a try during a weekday. Total(MB) Avg(kB) Name 2 4 /lib/ld-2.11.1.so 2 4 /lib/libbz2.so.1.0.4 2 4 /lib/libc-2.11.1.so 63 96 /lib/libcrypto.so.0.9.8 2 4 /lib/libdl-2.11.1.so 0 0 /lib/libnss_dns-2.11.1.so 0 0 /lib/libnss_files-2.11.1.so 2 4 /lib/libpthread-2.11.1.so 0 0 /lib/libresolv-2.11.1.so 2 4 /lib/librt-2.11.1.so 13 20 /lib/libssl.so.0.9.8 2 4 /lib/libz.so.1.2.3.3 2 4 /usr/lib/dovecot/imap 10 16 /usr/lib/dovecot/libdovecot-storage.so.0.0.0 2 4 /usr/lib/dovecot/libdovecot.so.0.0.0 2 4 /usr/lib/dovecot/modules/lib10_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib20_zlib_plugin.so 2 4 /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so 435 661 [anonymous] 640 973 [heap] 55 84 [stack] 1253 58352 TOTAL Quoting Timo Sirainen : > v2.1's imap/pop3 processes are linked with OpenSSL by default, > because of imapc/pop3c storages. I wonder if you notice any actual > (system-global) memory usage reduction when compiling without those > and you have thousands of imap processes? > > ./configure --with-storages=mdbox sdbox maildir mbox cydir > > I guess I could try it myself also but maybe someone else wanting to > reduce memory usage is quicker? :) > > Related to that, you can look at where imap processes' memory goes > with: http://dovecot.org/tools/linux-maps-parse-imap.pl > I don't know how much memory those (writable!) file-backed memory > areas actually use per-process. Maybe nothing, maybe something.. From carsten.delellis at delellis.net Sun Sep 16 13:48:17 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 12:48:17 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database Message-ID: Hi all I receive the following error in my mail.log. Does anybody know how to configure dovecot to get rid of this ? I have one server up and running with postfix, dovecot and a mysql backend. The error above happens on another server with postfix, dovecot and a ldap backend. Any hints ? Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From user+dovecot at localhost.localdomain.org Sun Sep 16 14:21:11 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Sun, 16 Sep 2012 13:21:11 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <5055B627.9010809@localhost.localdomain.org> On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > Hi all > > I receive the following error in my mail.log. > > Does anybody know how to configure dovecot to get rid of this ? > [?] Fix your setup. ,--[ http://wiki2.dovecot.org/UserDatabase ]-- | The user database lookup returns these fields: | ? | * home: User's home directory, overrides the global mail_home setting. | ? `-- See also: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Pascal -- The trapper recommends today: cafefeed.1226013 at localdomain.org From noel.butler at ausics.net Sun Sep 16 14:32:01 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:32:01 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <5055B627.9010809@localhost.localdomain.org> References: <5055B627.9010809@localhost.localdomain.org> Message-ID: <1347795121.3768.5.camel@tardis> thats a rather intelligent response now, isnt it troll. On Sun, 2012-09-16 at 13:21 +0200, Pascal Volk wrote: > On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > > Hi all > > > > I receive the following error in my mail.log. > > > > Does anybody know how to configure dovecot to get rid of this ? > > [?] > > Fix your setup. > > ,--[ http://wiki2.dovecot.org/UserDatabase ]-- > | The user database lookup returns these fields: > | ? > | * home: User's home directory, overrides the global mail_home setting. > | ? > `-- > > See also: http://wiki2.dovecot.org/VirtualUsers/Home > > > Regards, > Pascal -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Sun Sep 16 14:33:33 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:33:33 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <1347795213.3768.8.camel@tardis> Hi, On Sun, 2012-09-16 at 12:48 +0200, Carsten Laun-De Lellis wrote: > Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a ldap > backend. should use mysql :) -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From mostafa.rokooie at gmail.com Sun Sep 16 16:44:49 2012 From: mostafa.rokooie at gmail.com (Mostafa Rokooie) Date: Sun, 16 Sep 2012 18:14:49 +0430 Subject: [Dovecot] Edit an existing message or replacing it Message-ID: Hi, I want to know is there any solution to edit an existing message in mailbox via IMAP? or replace it with a new message (store a new message with same UID)? Thank you very much, --Mostafa Rokooie From carsten.delellis at delellis.net Sun Sep 16 17:30:36 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 16:30:36 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <1347795213.3768.8.camel@tardis> References: <1347795213.3768.8.camel@tardis> Message-ID: Hi First of all thank you for helping me to fix the problem. I am trying out ldap to get used to it. As I wrote I have a mysql setup up and running. Here's the output of doveconf -n: # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-028stab099.3 x86_64 Ubuntu 11.04 vzfs auth_mechanisms = plain login disable_plaintext_auth = no hostname = smtp.example.com mail_location = maildir:/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/vmail/%d/%n prefix = separator = / subscriptions = yes type = private } passdb { args = /scripts/ldap/mail/auth.ldap driver = ldap } plugin { recipient_delimiter = + sieve = /vmail/%d/%n/dovecot.sieve sieve_dir = /vmail/%d/%n sieve_global_dir = /var/lib/dovecot/sieve sieve_global_path = /var/lib/dovecot/sieve/default.sieve } postmaster_address = root at example.com protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0660 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl_cert = Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a > ldap backend. should use mysql :) From CMarcus at Media-Brokers.com Sun Sep 16 17:41:10 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 16 Sep 2012 10:41:10 -0400 Subject: [Dovecot] clucene build warnings? Message-ID: <5055E506.7040704@Media-Brokers.com> Hi all/Timo, Is clucene still the best/recommended method for enabling full text search? Although I haven't enabled FTS yet, I plan on doing sometime in the future, and was going to be using clucene, but before I do, I'd like to see if I need to worry about an issue I've been seeing ever since I first installed clucene (and still get every time it is upgraded)... I run gentoo linux on an amd64 server, and have always gotten this warning whenever it is updated (and also got it when I first installed it): QA Notice: Package triggers severe warnings which indicate that it may exhibit random runtime failures. /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? Please do not file a Gentoo bug and instead report the above QA issues directly to the upstream developers of this software. Homepage:http://clucene.sourceforge.net/ Has anyone else ever seen similar warnings when building clucene? Is the gentoo ebuild output correct that this is not a gentoo bug, but is in fact a problem with upstream? If so, is it harmless? Or should I try to get upstream to fix it? That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? Thanks, -- Best regards, Charles From sven at svenhartge.de Sun Sep 16 18:20:48 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 16 Sep 2012 17:20:48 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database References: <1347795213.3768.8.camel@tardis> Message-ID: <6947fe6hfsv8@mids.svenhartge.de> Carsten Laun-De Lellis wrote: > user_attrs = home=/vmail/%d/%n,=uid=5000,=gid=5000 This should be user_attrs = =home=/vmail/%d/%n,=uid=5000,=gid=5000 You need the = before "home" like you did with "uid" and "gid" because you are not assigning any LDAP attribute to home but are construction one yourself with %d and %n. Or you can use the default_fields stanza in your userdb configuration to provide those values. user_attrs would then be empty. Gr??e, Sven. -- Sigmentation fault. Core dumped. From lists at kokelnet.de Sun Sep 16 18:48:28 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Sun, 16 Sep 2012 17:48:28 +0200 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. Message-ID: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Hello list, I am using dovecot v2.1.9 and have a problem with doveadm backup: # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here my namespace config: namespace { list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/shared/%%u prefix = Gemeinsame-Postfaecher/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u prefix = separator = / subscriptions = yes type = private } Have I got a problem with my namespace config? Regards, Tobias Hachmer From slusarz at curecanti.org Sun Sep 16 21:55:39 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 16 Sep 2012 12:55:39 -0600 Subject: [Dovecot] Edit an existing message or replacing it In-Reply-To: References: Message-ID: <20120916125539.Horde.bh8AKIF5lbhQViCrQKfX2pA@bigworm.curecanti.org> Quoting Mostafa Rokooie : > I want to know is there any solution to edit an existing message in mailbox > via IMAP? or replace it with a new message (store a new message with same > UID)? You **can't** alter an existing UID message. You have to re-add the message to the mailbox and use a new UID. For this, you have 2 options: 1.) FETCH the message, alter it locally, APPEND the message to the server, and DELETE the original message. 2.) Use CATENATE to create a new message, and DELETE the original message (this is how we strip attachments from a message). Note that not all IMAP4rev1 servers support CATENATE. michael From dmalolepszy at optusnet.com.au Mon Sep 17 03:49:25 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 17 Sep 2012 10:49:25 +1000 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <20120913003845.GA10550@daniel.localdomain> References: <50501E34.2030900@optusnet.com.au> <20120913003845.GA10550@daniel.localdomain> Message-ID: <50567395.1040001@optusnet.com.au> On 13/09/12 10:38, Daniel Parthey wrote: > Dominic Malolepszy wrote: >> Is there a way to strip a backslash if it exists at the start of >> username like \user at example.com within the Dovecot LDA? I am using >> Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL >> trickery. > > Address rewriting should be done in the MTA. > I see no point why this should be done in the LDA? > > Regards > Daniel Agreed, I was just hoping I would not have to poke around the existing Sendmail setup we have here, in the end it turned out the backslashes were leftover legacy crud in some of the LDAP entries that actually did nothing. From crohmann at netcologne.de Mon Sep 17 09:52:37 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Mon, 17 Sep 2012 08:52:37 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <5056C8B5.3070706@netcologne.de> Hey dovecot-users, On 14.09.2012 16:59, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? I'd vote for OpenStack's Swift or Ceph's RADOS. They are both gaining momentum with new instrallations, they are open source and quite active in development. Also they both maintain Amazon S3 compatible APIs. Ceph even has a Swift compatible API for that matter. Regards Christian From alessio at skye.it Mon Sep 17 10:09:09 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 17 Sep 2012 09:09:09 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment Message-ID: <5056CC95.8030209@skye.it> Hi, I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. Dovecot's wiki suggest to enable acl like this: plugin { # assumes mailboxes are in /var/mail/%d/%n: acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db } but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. So, why don't provide a way to restrict shared mailboxes also for dict in SQL? One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. Are there other possible solutions? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tom at whyscream.net Mon Sep 17 11:40:11 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 17 Sep 2012 10:40:11 +0200 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <5056E1EB.5000600@whyscream.net> On 9/16/12 4:41 PM, Charles Marcus wrote: > Hi all/Timo, > > Is clucene still the best/recommended method for enabling full text search? > > Although I haven't enabled FTS yet, I plan on doing sometime in the > future, and was going to be using clucene, but before I do, I'd like to > see if I need to worry about an issue I've been seeing ever since I > first installed clucene (and still get every time it is upgraded)... > > I run gentoo linux on an amd64 server, and have always gotten this > warning whenever it is updated (and also got it when I first installed it): > > QA Notice: Package triggers severe warnings which indicate that it > may exhibit random runtime failures. > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: > warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, > int, size_t)? > Please do not file a Gentoo bug and instead report the above QA > issues directly to the upstream developers of this software. > Homepage:http://clucene.sourceforge.net/ > > Has anyone else ever seen similar warnings when building clucene? > > Is the gentoo ebuild output correct that this is not a gentoo bug, but > is in fact a problem with upstream? > > If so, is it harmless? Or should I try to get upstream to fix it? Same warnings here (gentoo 32bit). The gentoo package management simply checks the gcc output for warnings, and reports them to you after the build completes. As long as the warnings are not related to patches added by gentoo, the issue should go upstream. Whether it is a serious issue, that can only be decided by reviewing the code itself, which would be more appropriate for a lucene dev than a dovecot dev ;) I suggest that you simply report the issue upstream and see what happens. If the code is harmless, they might just change it the so that gcc doesn't complain about it in the future. -- Tom From CMarcus at Media-Brokers.com Mon Sep 17 12:56:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 05:56:35 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5056E1EB.5000600@whyscream.net> References: <5055E506.7040704@Media-Brokers.com> <5056E1EB.5000600@whyscream.net> Message-ID: <5056F3D3.4060006@Media-Brokers.com> On 2012-09-17 4:40 AM, Tom Hendrikx wrote: > I suggest that you simply report the issue upstream and see what > happens. If the code is harmless, they might just change it the so that > gcc doesn't complain about it in the future. Thanks Tom, but I already did, a long time ago (when I first installed it): https://sourceforge.net/tracker/?func=detail&aid=3494798&group_id=80013&atid=558446 Didn't hear anything for a month, and when I posted a follow-up, was told they were busy, and would take a look in the next few days, but nothing since and that was back in March... I just posted another follow-up, so we'll see what happens. I was mainly interested to know if anyone else sees this warning when compiling on other systems... -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 13:02:51 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:02:51 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) Message-ID: Hello, I have a quite common dovecot configuration with virtual users support: 1. User configuration is stored in MySQL database. 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. Is it possible to change mailbox mode to 0660? Dovecot version is 2.1.7 Part of SQL configuration: user_query = \ SELECT \ CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home, \ 500 AS uid, \ 500 AS gid \ FROM users u \ LEFT JOIN domains d ON u.domain_id = d.id \ WHERE \ u.user_name = '%n' AND \ d.domain_name = '%d' AND \ u.active = 1 Regards. -- M. From marcin at rootnode.net Mon Sep 17 13:09:22 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:09:22 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <2c3a5a4b367a9062389dfeb36d2cba61@rootnode.net> On 2012-09-17 12:02, Marcin H?ybin wrote: > 1. User configuration is stored in MySQL database. Obviously, I meant 0700 to be changed to 0770. -- M. From tss at iki.fi Mon Sep 17 15:21:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:21:51 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> On 16.9.2012, at 17.41, Charles Marcus wrote: > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? That's wrong, but it works anyway as intended. > That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? CLucene seems to be a pretty dead project. They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. From tss at iki.fi Mon Sep 17 15:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:36:32 +0300 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <55461F54-DC83-476B-910C-D8783E675DE8@iki.fi> On 13.9.2012, at 3.47, Daniel Parthey wrote: > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } UTF8 is (or should be) now used everywhere in config files. From tss at iki.fi Mon Sep 17 15:40:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:40:15 +0300 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. In-Reply-To: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> References: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Message-ID: <50602815-DD35-4B6B-B319-AA9261AC0812@iki.fi> On 16.9.2012, at 18.48, Tobias Hachmer wrote: > I am using dovecot v2.1.9 and have a problem with doveadm backup: > > # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir > doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Different installations use different kinds of namespaces for different purposes, so dsync can't simply sync all the namespaces. It syncs only the namespace which uses the global mail_location setting (not overriding it by having an explicit location setting). > namespace inbox { > inbox = yes > location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u So you probably should remove this location setting (and make sure mail_location is the same). > prefix = > separator = / > subscriptions = yes > type = private > } From CMarcus at Media-Brokers.com Mon Sep 17 15:42:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 08:42:32 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> Message-ID: <50571AB8.3010505@Media-Brokers.com> On 2012-09-17 8:21 AM, Timo Sirainen wrote: > CLucene seems to be a pretty dead project. Bummer... Was my memory correct that lucene is what you currently recommend for FTS for dovecot? > They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. Yuck - looks like it is java based? http://lucene.apache.org/ -- Best regards, Charles From tss at iki.fi Mon Sep 17 15:44:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:44:21 +0300 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <5056CC95.8030209@skye.it> References: <5056CC95.8030209@skye.it> Message-ID: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> On 17.9.2012, at 10.09, Alessio Cecchi wrote: > I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. > > Dovecot's wiki suggest to enable acl like this: > > plugin { > # assumes mailboxes are in /var/mail/%d/%n: > acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db > } > > but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. As long as all the servers have access to the file it doesn't matter where it is. You could even do something like /var/mail/nas1/shared-dict/%d.db > So, why don't provide a way to restrict shared mailboxes also for dict in SQL? > > One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. From tss at iki.fi Mon Sep 17 15:45:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:45:28 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50571AB8.3010505@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> Message-ID: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> On 17.9.2012, at 15.42, Charles Marcus wrote: > On 2012-09-17 8:21 AM, Timo Sirainen wrote: >> CLucene seems to be a pretty dead project. > > Bummer... > > Was my memory correct that lucene is what you currently recommend for FTS for dovecot? Lucene or Solr (i.e. anything but Squat). >> They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. > > Yuck - looks like it is java based? > > http://lucene.apache.org/ Both CLucene and Lucene++ are C++ ports of the original Java Lucene. From tss at iki.fi Mon Sep 17 15:46:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:46:38 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> On 17.9.2012, at 13.02, Marcin H?ybin wrote: > I have a quite common dovecot configuration with virtual users support: > 1. User configuration is stored in MySQL database. > 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. > > Is it possible to change mailbox mode to 0660? Yeah, although since they're already sharing UIDs I don't see much point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway describes how it all works. From tss at iki.fi Mon Sep 17 15:57:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:57:57 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> On 14.9.2012, at 18.16, Damien Churchill wrote: > On 14 September 2012 15:59, Timo Sirainen wrote: >> On 14.9.2012, at 17.46, Damien Churchill wrote: >> >>> I was wondering what would be entailed in modifying dovecot 2.2 to >>> support storing mail in an object store. I've seen a few mails dotted >>> around in the ML history about supporting such a thing and seen it's >>> basically dependant upon some changes in lib-storage to support >>> writing messages without locking. Is this still the case? >> >> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >> > > That's good to hear :) I've been evaluating RADOS as an object store, > which is similar to S3. Although any distributed storage would be > great. I'd be more than happy to help code it! I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > > I'm assuming that the director would be used in order to distribute > connections to the same server, so it's only within a local instance > of dovecot you'd need to be aware of what currently has a connection > open for that user? Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > How are you planning on handling the situation where say node X dies > and hasn't uploaded the latest index file? Would that result in > missing messages from the mailbox when accessed by another node, or is > the local index intended to be more of a write-through cache? No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) From marcin at rootnode.net Mon Sep 17 15:57:31 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 14:57:31 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <8b54646de4ef445102e8f40ee8115c83@rootnode.net> On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? I have a service running with different UID that must have full access to mail folders. UID/user is added to vmail group. That is why 0770 permission is desired. -- M. From marcin at rootnode.net Mon Sep 17 16:25:27 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:25:27 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway > describes how it all works. Already tested and it does not work. Directory structure is /home/mail/// drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support Two last directories for mailbox support at maildomain.com are created automatically after first e-mail comes to this address. As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). Maybe additional configuration is needed to make it work? -- M. From tss at iki.fi Mon Sep 17 16:30:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 16:30:59 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> On 17.9.2012, at 16.25, Marcin H?ybin wrote: > On 2012-09-17 14:46, Timo Sirainen wrote: >> Yeah, although since they're already sharing UIDs I don't see much >> point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway >> describes how it all works. > > Already tested and it does not work. > Directory structure is /home/mail/// > > drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home > drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail > drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support > > Two last directories for mailbox support at maildomain.com are created automatically > after first e-mail comes to this address. > > As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). > Maybe additional configuration is needed to make it work? What's your mail_location setting? Note the "Permissions to new user directories" in the wiki, especially the last paragraph. When Dovecot doesn't know what permissions it should use, it defaults to 0700 to be safe. From CMarcus at Media-Brokers.com Mon Sep 17 16:40:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 09:40:09 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> Message-ID: <50572839.9000301@Media-Brokers.com> On 2012-09-17 8:45 AM, Timo Sirainen wrote: > Both CLucene and Lucene++ are C++ ports of the original Java Lucene. Hmmm... ok, I guess I'm missing something... Googling lucene++ leads me to this page: http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: http://lucene.apache.org/ and on that page it says (among other things): "/Lucene Core /, our flagship sub-project, provides Java-based indexing and search technology, as well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities." Which seems to say that it is java based...? -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 16:40:35 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:40:35 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 2012-09-17 15:30, Timo Sirainen wrote: > What's your mail_location setting? Note the "Permissions to new user > directories" in the wiki, especially the last paragraph. When Dovecot > doesn't know what permissions it should use, it defaults to 0700 to > be > safe. mail_location = maildir:~/Maildir Maybe that's the point. What do you propose? I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" -- M. From CMarcus at Media-Brokers.com Mon Sep 17 17:37:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:37:19 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50572839.9000301@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <5057359F.8070107@Media-Brokers.com> Ok, just to wrap up this thread for anyone who cares, I just got a reply from the clucene dev (in response to my asking him about the possible merger of clucene and lucene++) after he said he had fixed this bug and the next version would contain the fix: "More or less it's true. About a year ago we started to make Lucene++ to the new CLucene version, as Lucene++ (also written in C++) is a port of a newer Apache Lucene version (written in Java) as the one CLucene is a port of. But we did not want to simply merge them, but to adapt Lucene++ to the "design principles" of CLucene. E.g., Lucene++ makes heavy use of shared pointers. And in CLucene we wanted to reduce this usage in favor of performance. But this not finished and I cannot say when it will finished. Nevertheless, the new version of CLucene (if any) will be also C++ and not Java. Best regards, Veit" On 2012-09-17 9:40 AM, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache > Lucene/Solr page: > > http://lucene.apache.org/ > > and on that page it says (among other things): > > "/Lucene Core /, our flagship > sub-project, provides Java-based indexing and search technology, as > well as spellchecking, hit highlighting and advanced > analysis/tokenization capabilities." > > Which seems to say that it is java based...? > -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From tomas at blueboard.cz Mon Sep 17 17:40:55 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Mon, 17 Sep 2012 16:40:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes Message-ID: <50573677.7060108@blueboard.cz> Hello, I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 I`d like to have more secure configuration > one process to one connection. Thanks for help. Tomas My config: # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 6200 default_process_limit = 4096 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /var/log/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 1536 service_count = 1 } service imap { process_limit = 2048 } service pop3-login { service_count = 1 } service pop3 { process_limit = 2048 } shutdown_clients = no ssl_cert = References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: On 17.9.2012, at 16.40, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: > > http://lucene.apache.org/ This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus From tss at iki.fi Mon Sep 17 17:46:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:46:46 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573677.7060108@blueboard.cz> References: <50573677.7060108@blueboard.cz> Message-ID: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> On 17.9.2012, at 17.40, Tom?? Randa wrote: > Hello, > > I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). > Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. From tss at iki.fi Mon Sep 17 17:47:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:47:50 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 17.9.2012, at 16.40, Marcin H?ybin wrote: > On 2012-09-17 15:30, Timo Sirainen wrote: >> What's your mail_location setting? Note the "Permissions to new user >> directories" in the wiki, especially the last paragraph. When Dovecot >> doesn't know what permissions it should use, it defaults to 0700 to be >> safe. > > mail_location = maildir:~/Maildir > > Maybe that's the point. What do you propose? > I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" Then your current possibilities are: a) mkdir + chmod the directory when a new user is created b) change Dovecot sources From CMarcus at Media-Brokers.com Mon Sep 17 17:52:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:52:36 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <50573934.1070909@Media-Brokers.com> On 2012-09-17 10:44 AM, Timo Sirainen wrote: > This is the only page I'm aware of for the project: > https://github.com/luceneplusplus/LucenePlusPlus Weird... wonder which is which, or if they are one and the same... Not much activity on that one either (most things are over a year old)... Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? -- Best regards, Charles From support at oeko.net Mon Sep 17 18:08:53 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 17:08:53 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota Message-ID: <20120917150852.GA13113@spruce.wiehl.oeko.net> Hi, I have now verified that there is a problem with quota (and sieve?). Summary: If the "maildirsize" file gets garbled (= every once in a while), mail is getting lost randomly. Details: I have a Debian box (6.0.5/amd64) with a Postfix server (2.7.1), which delivers email via Dovecot (1.2.15) to local disk (ext3). My email is filtered via a sieve script (wish I could have something better, eg. procmail). The account data is fetched from an SQL database, and I have a default quota set in dovecot.conf (200MB). My own mailbox has no quota (*:storage=0, mentioned in the database). A client sent an email to me, which is logged as being delivered like normal to my mailbox, but doesn't show up there. After poking around, I decided to simply delete the maildirsize file and have the client send that email again, and voila, it shows up in my mailbox. At the same time, however, other emails get delivered just fine, so the heinous thing is that you don't immediately notice that something fishy is going on - only when the client actually calls you (sort of defeats some uses of email)... The problem is worsened by the fact that you can't tell *which* emails get lost - you only end up with a random subset of "your" messages. The problem occurred now after running for a few months. FWIW, the problem with the garbled maildirsize file resulted in a different user's email getting bounced due to "mailbox full", when it wasn't. Questions: * Is this a known problem (Google was unhelpful/unwieldy)? * How can I prevent the problem, short of setting up a cron job to delete that file every week? * How can I log the delivery of messages in dovecot's log, not only the POP3/IMAP activity when people fetch their email? TIA! Kind regards, --Toni++ From tomas at blueboard.cz Mon Sep 17 18:10:55 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:10:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> Message-ID: <50573D7F.1040005@blueboard.cz> Hello, thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. Tomas R. Dne 17.09.12 16:46, Timo Sirainen napsal(a): > On 17.9.2012, at 17.40, Tom?? Randa wrote: > >> Hello, >> >> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 > That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. > From tss at iki.fi Mon Sep 17 18:16:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:16:20 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. On 17.9.2012, at 18.10, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >> > From tss at iki.fi Mon Sep 17 18:18:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:18:37 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50573934.1070909@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> <50573934.1070909@Media-Brokers.com> Message-ID: On 17.9.2012, at 17.52, Charles Marcus wrote: > On 2012-09-17 10:44 AM, Timo Sirainen wrote: >> This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus > > Weird... wonder which is which, or if they are one and the same... > > Not much activity on that one either (most things are over a year old)... > > Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? Some very large installations are successfully using it. I'm not planning on breaking it anytime soon. From f.bonnet at esiee.fr Mon Sep 17 18:21:05 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:21:05 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50573FE1.10906@esiee.fr> hello what is the result of the following command ? sysctl -a | grep kern.maxproc On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From f.bonnet at esiee.fr Mon Sep 17 18:22:16 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:22:16 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50574028.3070608@esiee.fr> and this one sysctl -a | grep kern.maxprocperuid On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From tomas at blueboard.cz Mon Sep 17 18:24:24 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:24:24 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <505740A8.3090803@blueboard.cz> On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) Thanks for your time Tomas R. Dne 17.09.12 17:16, Timo Sirainen napsal(a): > Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. > > On 17.9.2012, at 18.10, Tom?? Randa wrote: > >> Hello, >> >> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >> >> >> Tomas R. >> >> >> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>> >>>> Hello, >>>> >>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>> From tss at iki.fi Mon Sep 17 18:26:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:26:10 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <505740A8.3090803@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> Message-ID: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. On 17.9.2012, at 18.24, Tom?? Randa wrote: > On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: > > cputime unlimited > filesize unlimited > datasize 33554432 kbytes > stacksize 524288 kbytes > coredumpsize unlimited > memoryuse unlimited > vmemoryuse unlimited > descriptors 36864 > memorylocked unlimited > maxproc 8192 > sbsize unlimited > swapsize unlimited > > > Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) > > > Thanks for your time > > Tomas R. > > > Dne 17.09.12 17:16, Timo Sirainen napsal(a): >> Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. >> >> On 17.9.2012, at 18.10, Tom?? Randa wrote: >> >>> Hello, >>> >>> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >>> >>> >>> Tomas R. >>> >>> >>> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>>> >>>>> Hello, >>>>> >>>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>>> > From tss at iki.fi Mon Sep 17 18:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:45:05 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> Message-ID: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> On 17.9.2012, at 18.26, Timo Sirainen wrote: > It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: 1. dovecot master process starts, inheriting the ulimits from the shell that starts it 2. Dovecot checks the current limit and increases it if necessary 3. Forked child processes inherit the parent's ulimits 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. kern.maxprocperuid I guess means the upper allowed limit for the ulimit. From tomas at blueboard.cz Mon Sep 17 18:58:12 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:58:12 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> Message-ID: <50574894.4090609@blueboard.cz> kern.maxprocperuid is maximum allowed user limit, But in standard it is set to maximum imho. And if dovecot checks limit why it did not increases it on FreeBSD? :( ? Hard limit 8192 on my box is much more. Dne 17.09.12 17:45, Timo Sirainen napsal(a): > On 17.9.2012, at 18.26, Timo Sirainen wrote: > >> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. > > Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: > > 1. dovecot master process starts, inheriting the ulimits from the shell that starts it > 2. Dovecot checks the current limit and increases it if necessary > 3. Forked child processes inherit the parent's ulimits > 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) > > There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. > > kern.maxprocperuid I guess means the upper allowed limit for the ulimit. > From support at oeko.net Mon Sep 17 19:06:40 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 18:06:40 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota In-Reply-To: <20120917150852.GA13113@spruce.wiehl.oeko.net> References: <20120917150852.GA13113@spruce.wiehl.oeko.net> Message-ID: <20120917160640.GA16928@spruce.wiehl.oeko.net> Hi, On Mon, Sep 17, 2012 at 05:08:53PM +0200, Toni Mueller wrote: > I have now verified that there is a problem with quota (and sieve?). I was probably jumping to conclusions, as the delivery problem happens again - statistically extremely unlikely after this short while. kind regards, --Toni++ From kgc at corp.sonic.net Tue Sep 18 02:02:29 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 17 Sep 2012 16:02:29 -0700 Subject: [Dovecot] Proxy connection timeouts Message-ID: <5057AC05.2050904@corp.sonic.net> We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) I'm curious if anyone else has seen any similar problems or has any suggestions. # dovecot -n # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Scientific Linux release 6.3 (Carbon) auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes auth_verbose_passwords = sha1 auth_worker_max_count = 64 mail_fsync = always mail_log_prefix = "%s(%u): session=%{session} " mail_plugins = stats zlib maildir_very_dirty_syncs = yes mmap_disable = yes passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = imap driver = pam } plugin { lazy_expunge = DELETED_MESSAGES. stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 service anvil { client_limit = 10000 } service auth { client_limit = 10000 vsz_limit = 512 M } service doveadm { inet_listener { port = 1842 } unix_listener doveadm-server { mode = 0666 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 7000 process_min_avail = 32 } service imap-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-imap-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 4096 vsz_limit = 512 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 2000 process_min_avail = 32 } service pop3-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-pop3-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } shutdown_clients = no ssl = required ssl_ca = References: <50449193.8080101@um.es> Message-ID: <50581BCC.7050607@um.es> El 11/09/12 16:24, Timo Sirainen escribi?: > On 3.9.2012, at 14.16, Angel L. Mateo wrote: > >> Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): >> >> doveadm auth cache flush >> 4 cache entries flushed > > Yep. > >> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) > > Works in my tests. > Is this cache the same than the user information cache? The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: doveadm auth cache flush in the logs I get: Sep 18 08:47:13 myotis34 dovecot: auth: Debug: master in: CACHE-FLUSH#0111#011 now I ask for user information with: doveadm user and this is what I get from logs: Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:19 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:19 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:41 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:41 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 As you can see in the third message, it is still using information from userdb cache From alessio at skye.it Tue Sep 18 10:50:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 18 Sep 2012 09:50:30 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> References: <5056CC95.8030209@skye.it> <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> Message-ID: <505827C6.6080204@skye.it> Il 17/09/2012 14:44, Timo Sirainen ha scritto: >> So, why don't provide a way to restrict shared mailboxes also for dict in SQL? >> >> One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. > If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. Good, after change my shared namespace from namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } to namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ ## <<=- change %u to %n separator = / subscriptions = no type = shared } shared mailboxes have become available only inside the same domains. Thanks! -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From ap00 at mail.ru Tue Sep 18 14:04:31 2012 From: ap00 at mail.ru (=?UTF-8?B?0JDQu9C10LrRgdC10Lkg0J/QsNC90LrRgNCw0YLQvtCy?=) Date: Tue, 18 Sep 2012 15:04:31 +0400 Subject: [Dovecot] =?utf-8?q?lda_bug=3A_multiple_passdb_ignored?= Message-ID: <1347966271.760993075@f26.mail.ru> Greetings. When dovecot configured to use multiple passdb and static userdb as: #password database in multiple ldap subtree passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-real.conf } passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-virtual.conf } userdb { ?driver = static ?args = uid=pop gid=mailnull } all works fine but... When LDA deliver mail it check passdb but only first appeared in configuration: 18.09.2012 15:00:18 auth: Debug: master in: USER??????? 1?????? info at zzz.zz ???? service=lda 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword 18.09.2012 15:00:18 lda: Debug: auth input: 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND?? 1 1 P.S. dovecot version 2.1.7 sysytem x86 FreeBSD 9.0 From janfrode at tanso.net Tue Sep 18 14:38:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 18 Sep 2012 13:38:53 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? Message-ID: We have a sieve script doing sieve_before to sort spam to spam-folders. Now I'm trying to configure the "Trash" plugin, but it doesn't seem to work.. I noticed my config file says: # Space separated list of plugins to load (none known to be useful so far). Do NOT # try to load IMAP plugins here. #mail_plugins = and that doveconf doesn't list any plugins loaded for "protocol sieve". Should we load quota and trash here ? -jf From Jost.Krieger+dovecot at rub.de Tue Sep 18 14:40:12 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 18 Sep 2012 13:40:12 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) Message-ID: <20120918114011.GH8536@ruhr-uni-bochum.de> It seems there is no way to correct te quota limit downward for some mailboxes, like: quota_rule9 = UCE-TMP:storage=-10%% The idea is to not eat up the users storage with spam, so that spam stays in the queue (for some time) while real mail gets delivered. There *is* a line } else if (*value != '-' && relative_rule) { in quota.c, but nothing else handles it, starting with quota_rule_parse_percentage. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From stephan at rename-it.nl Tue Sep 18 16:41:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 15:41:16 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? In-Reply-To: References: Message-ID: <505879FC.4010907@rename-it.nl> On 9/18/2012 1:38 PM, Jan-Frode Myklebust wrote: > We have a sieve script doing sieve_before to sort spam to > spam-folders. Now I'm trying to configure the "Trash" plugin, but it > doesn't seem to work.. I noticed my config file says: > > # Space separated list of plugins to load (none known to be useful > so far). Do NOT > # try to load IMAP plugins here. > #mail_plugins = > > and that doveconf doesn't list any plugins loaded for "protocol > sieve". Should we load quota and trash here ? Keep in mind that the Sieve language and the Sieve protocol are two different things. The Sieve language is what processes incoming e-mails; for Dovecot it is executed by a plugin for LTMP or LDA during message delivery. The Sieve protocol - which is actually called ManageSieve - is used to upload Sieve scripts to the Server. That mail_plugins setting applies to the Sieve protocol, for which currently no useful plugins are known. The Sieve protocol service does not (currently) open any mail stores, which makes configuring mail-related plugins for it - such as quota and trash - utterly useless. So, in short, the answer to the above question is no. To help you further, we need some more information about your configuration. First of all, please provide the output of dovecot -n to this list. Also, describe more thoroughly what is going wrong. Do you see any log messages, etc. Regards, Stephan. From stephan at rename-it.nl Tue Sep 18 17:34:51 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 16:34:51 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058868B.8080904@rename-it.nl> Hello Dovecot users, I've been busy recently with some wholly unrelated Dovecot IMAP patches, so Sieve development moved a bit into the background. This release therefore consists mostly of small bugfixes. The most important change is the fix of a bug in ManageSieve's CHECKSCRIPT command that causes a segfault. Quite a few people have encountered this problem recently, so it is about time I release this now. Changelog v0.3.2: + sieve-refilter tool: improved man page documentation by explicitly specifying the syntax used for mailbox arguments. + Sieve: spamtest and virustest extensions: improved trace debugging of score calculation. + Sieve: made error messages about exceeding the maximum number of actions more verbose. - Sieve tools: fixed problems with running as root: sievec and sieve-dump now ignore mail_uid and mail_gid settings when run as root. - Sieve: fixed bug in action accounting (for limit checking): increase action instance count only when an action is actually created. - Sieve: include extension: fixed namespace separation of :global and :personal scripts. - ManageSieve: fixed segfault bug triggered by CHECKSCRIPT command. - Fixed linking with ld.gold. - Fixed several Clang compile warnings and a few potential bugs. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Tue Sep 18 18:48:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 18:48:19 +0300 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: This is now implemented as a plugin in hg. Usage in the commit message: http://hg.dovecot.org/dovecot-2.1/rev/f5bb9f6b304d Also there's a generic change where symlinks within same directory are treated as mailbox aliases, which aren't counted towards quota. On 2.9.2012, at 18.32, Timo Sirainen wrote: > I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. > > So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: > > .. > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > .. > > If "Sent Messages" doesn't exist: > - DELETE Sent works as usual > - RENAME Sent works as usual > > If "Sent Messages" is created: > - it's symlinked to Sent > - DELETE "Sent messages" deletes the symlink, not the contents > - RENAME "Sent messages" isn't allowed > - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. > > Thoughts? > > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Tue Sep 18 19:04:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:04:06 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > passdb { > driver = ldap > args = /usr/local/etc/dovecot/dv-ldap-virtual.conf > } Does dv-ldap-virtual.conf have auth_bind=yes? From tss at iki.fi Tue Sep 18 19:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:17:50 +0300 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120918114011.GH8536@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: On 18.9.2012, at 14.40, Jost Krieger wrote: > It seems there is no way to correct te quota limit downward for some > mailboxes, like: > > quota_rule9 = UCE-TMP:storage=-10%% > > The idea is to not eat up the users storage with spam, so that spam > stays in the queue (for some time) while real mail gets delivered. Interesting idea. This was pretty easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. From tss at iki.fi Tue Sep 18 19:31:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:31:08 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50581BCC.7050607@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> Message-ID: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >> >> Works in my tests. >> > Is this cache the same than the user information cache? Yes. > The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: > > doveadm auth cache flush What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: doveadm auth cache flush foo # make sure it isn't there doveadm user foo doveadm auth cache flush foo Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. From tss at iki.fi Tue Sep 18 19:34:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:34:49 +0300 Subject: [Dovecot] Proxy connection timeouts In-Reply-To: <5057AC05.2050904@corp.sonic.net> References: <5057AC05.2050904@corp.sonic.net> Message-ID: <15C8389A-3C3A-461B-8FF8-FBACBDDB6A6C@iki.fi> On 18.9.2012, at 2.02, Kelsey Cummings wrote: > We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. > > dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) > > I'm curious if anyone else has seen any similar problems or has any suggestions. I once had similar problems when the proxy backend was Courier. The problems went away after migration to Dovecot was complete. The possibilities are either: a) The backend server is busy and doesn't have a chance to accept() the connection. b) Packets get dropped in the network and the retry packet is slow in coming (or also gets lost). Changing some kernel settings might help with a). There are also kernel settings that specify how SYN resend is attempted, you could try reducing it to a few seconds. From ap00 at mail.ru Tue Sep 18 19:40:39 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 20:40:39 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> References: <1347966271.760993075@f26.mail.ru> <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> Message-ID: <106-722288046.20120918204039@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:04:06 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> passdb { >> driver = ldap >> args = /usr/local/etc/dovecot/dv-ldap-virtual.conf >> } TS> Does dv-ldap-virtual.conf have auth_bind=yes? No. All ldap connection configured to use one prescribed ldap binding. P.S. Thank you for your work. -- Best regards, Anthony mailto:ap00 at mail.ru From tss at iki.fi Tue Sep 18 19:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:58:53 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > When dovecot configured to use multiple passdb and static userdb as: .. > all works fine but... > > When LDA deliver mail it check passdb but only first appeared in configuration: > > 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda > 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss > 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword > 18.09.2012 15:00:18 lda: Debug: auth input: > 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: > 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 From robert at schetterer.org Tue Sep 18 20:47:09 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 18 Sep 2012 19:47:09 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <5058B39D.5030501@schetterer.org> Am 18.09.2012 18:17, schrieb Timo Sirainen: > On 18.9.2012, at 14.40, Jost Krieger wrote: > >> It seems there is no way to correct te quota limit downward for some >> mailboxes, like: >> >> quota_rule9 = UCE-TMP:storage=-10%% >> >> The idea is to not eat up the users storage with spam, so that spam >> stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf > > Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. > some allready existing way is i.e with milter and postfix tagged spam get in hold queue for i.e human admin investigate, or some quarantaine with amavis for sure there are benefits getting dovecot involved in spam tagging but i would leave spam tagging to the mailserver in first way and deliver it to Junk folder by global sieve rule with giving more quota or ignoring Quota to Junk folder, but i see this isnt the same what you might wanna goal -- Best Regards MfG Robert Schetterer From jcblanco at fi.upm.es Tue Sep 18 21:18:21 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 18 Sep 2012 20:18:21 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058BAED.5080505@fi.upm.es> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the new pigeonhole version, however I've got the follwing errors when compiling: gcc -shared .libs/rfc2822.o .libs/edit-mail.o .libs/sieve-settings.o .libs/sieve-message.o .libs/sieve-smtp.o .libs/sieve-lexer.o .libs/sieve-script.o .libs/sieve-script-file.o .libs/sieve-script-dict.o .libs/sieve-ast.o .libs/sieve-binary.o .libs/sieve-binary-file.o .libs/sieve-binary-code.o .libs/sieve-binary-debug.o .libs/sieve-parser.o .libs/sieve-address.o .libs/sieve-validator.o .libs/sieve-generator.o .libs/sieve-interpreter.o .libs/sieve-runtime-trace.o .libs/sieve-code-dumper.o .libs/sieve-binary-dumper.o .libs/sieve-result.o .libs/sieve-error.o .libs/sieve-objects.o .libs/sieve-stringlist.o .libs/sieve-comparators.o .libs/sieve-match-types.o .libs/sieve-address-parts.o .libs/sieve-match.o .libs/sieve-commands.o .libs/sieve-code.o .libs/sieve-actions.o .libs/sieve-extensions.o .libs/sieve-plugins.o .libs/cmp-i-octet.o .libs/cmp-i-ascii-casemap.o .libs/mcht-is.o .libs/mcht-contains.o .libs/mcht-matches.o .libs/tst-truefalse.o .libs/tst-not.o .libs/tst-anyof.o .libs/tst-allof.o .libs/tst-address.o .libs/tst-header.o .libs/tst-exists.o .libs/tst-size.o .libs/cmd-require.o .libs/cmd-stop.o .libs/cmd-if.o .libs/cmd-keep.o .libs/cmd-redirect.o .libs/cmd-discard.o .libs/ext-fileinto.o .libs/ext-reject.o .libs/ext-envelope.o .libs/ext-encoded-character.o .libs/sieve.o -Wl,--whole-archive ../../src/lib-sieve/plugins/vacation/.libs/libsieve_ext_vacation.a ../../src/lib-sieve/plugins/subaddress/.libs/libsieve_ext_subaddress.a ../../src/lib-sieve/plugins/comparator-i-ascii-numeric/.libs/libsieve_ext_comparator-i-ascii-numeric.a ../../src/lib-sieve/plugins/relational/.libs/libsieve_ext_relational.a ../../src/lib-sieve/plugins/regex/.libs/libsieve_ext_regex.a ../../src/lib-sieve/plugins/copy/.libs/libsieve_ext_copy.a ../../src/lib-sieve/plugins/imap4flags/.libs/libsieve_ext_imap4flags.a ../../src/lib-sieve/plugins/include/.libs/libsieve_ext_include.a ../../src/lib-sieve/plugins/body/.libs/libsieve_ext_body.a ../../src/lib-sieve/plugins/variables/.libs/libsieve_ext_variables.a ../../src/lib-sieve/plugins/enotify/.libs/libsieve_ext_enotify.a ../../src/lib-sieve/plugins/notify/.libs/libsieve_ext_notify.a ../../src/lib-sieve/plugins/environment/.libs/libsieve_ext_environment.a ../../src/lib-sieve/plugins/mailbox/.libs/libsieve_ext_mailbox.a ../../src/lib-sieve/plugins/date/.libs/libsieve_ext_date.a ../../src/lib-sieve/plugins/spamvirustest/.libs/libsieve_ext_spamvirustest.a ../../src/lib-sieve/plugins/ihave/.libs/libsieve_ext_ihave.a ../../src/lib-sieve/plugins/editheader/.libs/libsieve_ext_editheader.a ../../src/lib-sieve/plugins/vnd.dovecot/debug/.libs/libsieve_ext_debug.a ../../src/lib-sieve/plugins/vnd.dovecot/duplicate/.libs/libsieve_ext_duplicate.a -Wl,--no-whole-archive -Wl,--rpath -Wl,/usr/lib64/dovecot -L/var/tmp/dovecot-2.1-pigeonhole-0.3.2-219_3-root/usr/lib64/dovecot -L/usr/lib64/dovecot -ldovecot-lda -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-storage/.libs -L/usr/kerberos/lib64 -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-dovecot/.libs -lrt -m64 -mtune=generic -Wl,-soname -Wl,libdovecot-sieve.so.0 -o .libs/libdovecot-sieve.so.0.0.0 /usr/bin/ld: cannot find -ldovecot-lda collect2: ld returned 1 exit status libtool: install: error: relink `libdovecot-sieve.la' with the above command before installing it make[4]: *** [install-dovecot_pkglibLTLIBRARIES] Error 1 make[4]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src' make: *** [install-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.50565 (%install) It seems that the error is related to the ld.gold fix, If I revert the c52a0c561311 patch the error is not produced Regards From stephan at rename-it.nl Tue Sep 18 21:49:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 20:49:19 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058BAED.5080505@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> Message-ID: <5058C22F.1070704@rename-it.nl> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: > Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the > new pigeonhole version, however I've got the follwing errors when > compiling: > > It seems that the error is related to the ld.gold fix, If I revert the > c52a0c561311 patch the error is not produced Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, but I forgot the Dovecot v2.1 tree. Fixed now: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c This only happens when you compile against installed Dovecot headers, people using --with-dovecot= (like me) would not have seen this. I'll push out a new release once 2.1.10 comes out. I'm being told that is imminent. Regards, Stephan. From tss at iki.fi Tue Sep 18 21:52:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 21:52:27 +0300 Subject: [Dovecot] v2.1.10 released Message-ID: <1347994347.13571.67.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig The director bugfix is pretty important at least when running in two director setups, since it can break the directors until both of them are manually shut down. I'm not entirely sure if the same happens in >2 director setups, but I'd think so (although I do wonder why I haven't seen it happen in them). BTW. I'm hoping for v2.2.alpha1 release within a week or two and to get it stabilized rather quickly. + imap: Implemented THREAD=ORDEREDSUBJECT extension. + Added "doveadm exec" command to easily execute commands from libexec_dir, e.g. "doveadm exec imap -u user at domain" + Added "doveadm copy" command. + doveadm copy/move: Added optional user parameter to specify the source username. This allows easily copying mails between different users. + Added namespace { disabled } setting to quickly enable/disable namespaces. This is especially useful when its value is returned by userdb. + Added mailbox_alias plugin. It allows creating mailbox aliases using symlinks. + imapc storage: Added imapc_max_idle_time setting to force activity on connection. + fts-solr: Expunging multiple messages is now faster. - director: In some conditions director may have disconnected from another director (without logging about it), thinking it was sending invalid data. - imap: Various fixes to listing mailboxes. - pop3-migration plugin: Avoid disconnection from POP3 server due to idling. - login processes crashed if there were a lot of local {} or remote {} settings blocks. From ap00 at mail.ru Tue Sep 18 22:04:42 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 23:04:42 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> References: <1347966271.760993075@f26.mail.ru> <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> Message-ID: <186-713645281.20120918230442@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:58:53 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> When dovecot configured to use multiple passdb and static userdb as: TS> .. >> all works fine but... >> >> When LDA deliver mail it check passdb but only first appeared in configuration: >> >> 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda >> 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss >> 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword >> 18.09.2012 15:00:18 lda: Debug: auth input: >> 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: >> 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 TS> This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 Thank you very much. -- Best regards, Anthony mailto:ap00 at mail.ru From stephan at rename-it.nl Tue Sep 18 22:36:34 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 21:36:34 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.3 for Dovecot v2.1.10 Message-ID: <5058CD42.3000507@rename-it.nl> Hello Dovecot users, Version 0.3.2 was a bit broken, because it failed to compile against installed Dovecot headers. Changelog v0.3.3: - Fixed compile against installed Dovecot headers. This was broken by the ld.gold fix in the previous release. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From interfasys at gmail.com Tue Sep 18 23:32:18 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 21:32:18 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) Message-ID: <5058DA52.7070400@gmail.com> Hello, There is a big problem with 2.1.10. Here is what I saw in the logs: Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed with signal 6 (core dumped) Reverting to 2.1.9 with the exact same config file works just fine Cheers, Olivier From Ralf.Hildebrandt at charite.de Tue Sep 18 23:34:03 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 18 Sep 2012 22:34:03 +0200 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <20120918203403.GK15878@charite.de> * interfaSys s?rl : > Hello, > > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine So far I'm not having problems wit 2.1.10 here (coming from 2.1.9) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 18 23:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 23:42:57 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> On 18.9.2012, at 23.32, interfaSys s?rl wrote: > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine gdb backtrace? http://dovecot.org/bugreport.html From interfasys at gmail.com Wed Sep 19 00:18:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 22:18:07 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> Message-ID: <5058E50F.10000@gmail.com> I thought it would be easily identifiable and a common problem, but apparently not :) Here is the trace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at user.c:17 _module_tmp = (void *) 0x10a84d28 asu = (struct antispam_user *) 0x10a84d28 tmp = 0x0 #9 0x00000000104f1f04 in hook_mail_user_created () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #10 0x00000000104efaf9 in mail_user_init () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000010495858 in mail_storage_service_init_post.46586.6405 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000010495582 in mail_storage_service_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #13 0x00000000104951b3 in mail_storage_service_lookup_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #14 0x000000000041f112 in client_create_from_input (input=0x7fffffffe940, login_client=0x10a550c0, fd_in=11, fd_out=11, input_buf=0x7fffffffe900, error_r=0x7fffffffe938) at main.c:202 user = (struct mail_storage_service_user *) 0x10a55141 mail_user = (struct mail_user *) 0x3 client = (struct client *) 0x7fffffffec70 set = (const struct imap_settings *) 0x7fffffffe900 flags = 32767 #15 0x000000000041eeaa in login_client_connected (client=0x10a550c0, username=0x10a04093 "test at example.com", extra_fields=0x10a04118) at main.c:272 input = {module = 0x423d0f "imap", service = 0x423d0f "imap", username = 0x10a04093 "test at example.com", session_id = 0x10a55120 "UCFMXQDK5gBewxZf", local_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "N.8Q", '\0' , __u6_addr16 = {11854, 20792, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1362636366, 0, 0, 0}}}, ip4 = {s_addr = 1362636366}}}, remote_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "^?\026_", '\0' , __u6_addr16 = {50014, 24342, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1595327326, 0, 0, 0}}}, ip4 = {s_addr = 1595327326}}}, local_port = 0, remote_port = 0, userdb_fields = 0x10a04118, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} error = (const __unknown__ *) 0x0 input_buf = {data = 0x10a55141, used = 3, priv = {0x0, 0x0, 0x3, 0x0, 0x0}} #16 0x00000000105a229f in master_login_auth_finish.11302 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x00000000105efa50 in master_login_auth_callback.11253.2275 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #18 0x0000000010596f56 in master_login_auth_input_user.11520 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x000000001058bf99 in master_login_auth_input.11492 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00000000105e9b34 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x00000000105d1ba7 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #22 0x00000000105d1a55 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x00000000105d1790 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #24 0x000000000041ed5d in main (argc=1, argv=0x7fffffffec60) at main.c:389 set_roots = {0x424020, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ed9d , failure_callback = 0x4194ee } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x424020, 0x0} Cheers, Olivier On 18/09/2012 21:42, Timo Sirainen wrote: > On 18.9.2012, at 23.32, interfaSys s?rl wrote: > >> There is a big problem with 2.1.10. >> >> Here is what I saw in the logs: >> Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 >> (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) >> Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed >> with signal 6 (core dumped) >> >> Reverting to 2.1.9 with the exact same config file works just fine > > gdb backtrace? http://dovecot.org/bugreport.html > > > From tss at iki.fi Wed Sep 19 02:42:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 02:42:34 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <1CB6DE70-A784-4D90-8127-BF87EDA84407@iki.fi> On 19.9.2012, at 0.18, interfaSys s?rl wrote: > I thought it would be easily identifiable and a common problem, but > apparently not :) .. > #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at > user.c:17 Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. From robert at schetterer.org Wed Sep 19 10:40:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 09:40:29 +0200 Subject: [Dovecot] v2.1.10 released In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <505976ED.4090602@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > + Added mailbox_alias plugin. It allows creating mailbox aliases using > symlinks. Hi Timo thx for coding, no Problems so far, after update, but mailbox_alias plugin should be described at http://wiki2.dovecot.org/Plugins/ -- Best Regards MfG Robert Schetterer From bangkokmaco at gmail.com Wed Sep 19 11:00:55 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 19 Sep 2012 15:00:55 +0700 Subject: [Dovecot] ssl cert for mail server Message-ID: for testing a new ssl cert. it works ok for browsers, but > openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 => verify error:num=19:self signed certificate in certificate chain is this ssl cert - as it's constructed - is ok for mail clients? (realize needs to be on mail port etc - right now talking about the cert itself.) have had problems with thunderbird, and was wondering if this might be part of the problem. From amateo at um.es Wed Sep 19 11:22:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 10:22:42 +0200 Subject: [Dovecot] Deleting a folder with & character Message-ID: <505980D2.3060401@um.es> Hello, One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). I can see the maildir with a doveadm list command: amateo_adm at myotis31:~$ sudo doveadm mailbox list -u ... INBOX.Kron & SPM ... but I can't delete it, neithe rename it: amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? From acrow at integrafin.co.uk Wed Sep 19 11:29:12 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Wed, 19 Sep 2012 09:29:12 +0100 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <50598258.1030005@integrafin.co.uk> On 19/09/12 09:22, Angel L. Mateo wrote: > Hello, > > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe > created a long time ago, when we use courier imap as pop/imap server, > but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox > doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox > doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a > specific syntax or escape to indicate the name? > Hi, I see something similar when I try to doveadm import maildir folders (to mdbox) with ampersands - I get the same "Mailbox doesn't exist: INBOX.Foo &- Bar" with that extraneous hyphen. Cheers Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. "Transact" is operated by Integrated Financial Arrangements plc Domain House, 5-7 Singer Street, London EC2A 4BQ Tel: (020) 7608 4900 Fax: (020) 7608 5300 (Registered office: as above; Registered in England and Wales under number: 3727592) Authorised and regulated by the Financial Services Authority (entered on the FSA Register; number: 190856) From tss at iki.fi Wed Sep 19 12:14:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 12:14:07 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> On 19.9.2012, at 11.22, Angel L. Mateo wrote: > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). From amateo at um.es Wed Sep 19 12:15:53 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 11:15:53 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <50598D49.8060507@um.es> El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > I have made the test in my test server (it has no real activity). In this server, user entry is refreshed correctly. But the same test in my production servers fails. I have checked (in the production one) that the second flush delete entries (in fact, 2, not 1): amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M but quota information is not reloaded from ldap server. I have also checked my ldap server, and dovecot is not performing any search operation for the user after flushing him from the cache. I have attached my doveconf -n. In my ldap configuration I have: user_attrs = irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule pass_attrs = irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule and I have check that my test and production server has the same configuration (but client_limit and number of processes). Any idea? From c at roessner-network-solutions.com Wed Sep 19 12:26:33 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 11:26:33 +0200 Subject: [Dovecot] OT: z-Push Message-ID: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Hi, just a short question: I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. Also I can not remove that folder over ActiveSync. Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 Thanks in advance -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From robert at schetterer.org Wed Sep 19 12:42:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 11:42:12 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: <50599374.4010902@schetterer.org> Am 19.09.2012 11:26, schrieb Christian R??ner: > Hi, > > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? yes it works with dovecot > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 sorry tested , there is no option for create folder on my android, need test account offlist? > > Thanks in advance > > -Christian R??ner > > --- > Bachelor of Science Informatik > Erlenwiese 14, 36304 Alsfeld > T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 > USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com > -- Best Regards MfG Robert Schetterer From CMarcus at Media-Brokers.com Wed Sep 19 13:36:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 19 Sep 2012 06:36:12 -0400 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> Message-ID: <5059A01C.2080706@Media-Brokers.com> On 2012-09-19 5:14 AM, Timo Sirainen wrote: > It's an invalid mailbox name and you can't access it using doveadm. > Either rename it with "mv" or via IMAP protocol (not entirely sure if > it works via IMAP either). Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? Thanks Timo, -- Best regards, Charles From alessio at skye.it Wed Sep 19 13:54:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 12:54:33 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <5059A469.6060604@skye.it> Hi, I have found this strange problem. I'm working with Debian 6, dovecot 2.1.9 and vpopmail-auth. LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. If disable ACL (acl and imap_acl from mail_plugins) in dovecot.conf "Segmentation fault" go away. Moreover, without ACL at the first delivery folders like Sent, Drafts, Trash, Spam appear immediately. An example, new user: root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? maildirsize ??? new ??? tmp root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Segmentation fault root at demo-vpop ~ # echo $? 139 root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? dovecot-acl-list ??? dovecot.index.cache ??? dovecot.index.log ??? dovecot-uidlist ??? dovecot-uidvalidity ??? dovecot-uidvalidity.50599a7f ??? maildirsize ??? new ? ??? 1348049535.M21758P19264.demo-vpop.cbsolt.net,S=2025,W=2071 ??? tmp dovecot log (with ulimit -c unlimited before starting Dovecot): Sep 19 12:06:50 auth: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:06:50 auth-worker(18728): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth-worker(18728): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:06:50 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 dict: Info: mysql(localhost): Connected to database dovecot lda log: Sep 19 12:06:50 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:06:50 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found now works fine: root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Sep 19 12:08:18 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:08:18 auth-worker(18763): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:08:18 auth-worker(18763): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:08:18 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 dict: Info: mysql(localhost): Connected to database dovecot Sep 19 12:08:18 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:08:18 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M } protocols = imap pop3 sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service pop3-login { service_count = 0 } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From florob at babelmonkeys.de Wed Sep 19 14:56:26 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 13:56:26 +0200 Subject: [Dovecot] ssl cert for mail server In-Reply-To: References: Message-ID: <5059B2EA.3040605@babelmonkeys.de> Am 19.09.2012 10:00, schrieb cc "maco" young: > for testing a new ssl cert. it works ok for browsers, but > >> openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 > > => verify error:num=19:self signed certificate in certificate chain > > is this ssl cert - as it's constructed - is ok for mail clients? (realize > needs to be on mail port etc - right now talking about the cert itself.) > have had problems with thunderbird, and was wondering if this might be > part of the problem. > Hi, first of all this is likely off topic for this ML, I'll still answer though, since I'm always intrigued by TLS problems. The reason openssl doesn't accept this cert, while your browser does, is quite likely that your system wide accepted CAs don't include Starfield Technologies, while your browser's CAs do (This is the case for Firefox and Thunderbird). However, I suspect that your mail addresses are of the form , and ms1.trailsandtribulations.net is what is in your MX record. As such the certificate needs to be valid for trailsandtribulations.net, and not ms1.trailsandtribulations.net. So you either need trailsandtribulations.net as your CN, or a SAN of type DNSName for trailsandtribulations.net. Cf. https://tools.ietf.org/html/rfc6125 for best practices on generating certificates. Regards, Florian From ml at cibest.com Wed Sep 19 15:15:48 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 19 Sep 2012 14:15:48 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <50504ED3.5030601@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> Message-ID: <5059B774.6060808@cibest.com> On 12/09/2012 10:58, Cibest ML wrote: > On 11/09/2012 20:48, Timo Sirainen wrote: >> On 8.9.2012, at 18.07, Cibest ML wrote: >> >>> - Message delivered by LMTP >>> Quota name Type Value Limit >>> User quota STORAGE 2653 524288 >>> User quota MESSAGE 1 - >>> >>> - quota recalc >>> Quota name Type Value Limit >>> User quota STORAGE 5307 524288 >>> User quota MESSAGE 2 - >> Quota recalculation is clearly seeing the message twice. My first >> guess was that you had two namespaces pointing to the same location, >> but you seem to have only one. Do you have symlinks in your Maildir? >> > Hi, > Nothing specific on the namespace side. > The only entry is the default one in 10-mail.conf > namespace inbox { > inbox = yes > } > > I tried to comment it out but it doesn't change anything > > And no symlinks in the Maildir. > Here's the file structure of Maildir for the test mailbox > > ./dovecot-uidvalidity.504df4e7 > ./dovecot-uidvalidity > ./.Sent > ./.Sent/maildirfolder > ./.Sent/new > ./.Sent/dovecot.index.log > ./.Sent/tmp > ./.Sent/dovecot-uidlist > ./.Sent/cur > ./.Spam > ./.Spam/maildirfolder > ./.Spam/new > ./.Spam/dovecot.index.log > ./.Spam/tmp > ./.Spam/dovecot-uidlist > ./.Spam/cur > ./new > ./subscriptions > ./.Drafts > ./.Drafts/maildirfolder > ./.Drafts/new > ./.Drafts/dovecot.index.log > ./.Drafts/tmp > ./.Drafts/dovecot-uidlist > ./.Drafts/cur > ./.Drafts/dovecot-keywords > ./dovecot.index.log > ./tmp > ./dovecot.mailbox.log > ./dovecot-uidlist > ./.Trash > ./.Trash/maildirfolder > ./.Trash/new > ./.Trash/dovecot.index.log > ./.Trash/tmp > ./.Trash/dovecot-uidlist > ./.Trash/cur > ./cur > ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa > ./dovecot-keywords > ./dovecot.index.cache > > -- > Regards > S?bastien We did some more investigations as we had to enable the no enforcing quota option to prevent too many compltains from our customers. As the issue seems to be linked to the inbox folder, we tried to focus on that. According to Timo's last post, that could be linked to namespace or symlinks (we don't have any). There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). Now doing some check via telnet doesn't show any mysterious "Inbox" list "" "*" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Spam" * LIST (\HasNoChildren) "." "Drafts" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "INBOX" lsub "" "*" * LSUB () "." "Trash" * LSUB () "." "Sent" * LSUB () "." "Spam" * LSUB () "." "Drafts" * LSUB () "." "Inbox" 04 select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. 04 select Inbox * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. It's working for both but I checked on other working dovecot servers, that seems to be a normal behavior for Inbox folder I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) -- S?bastien From tss at iki.fi Wed Sep 19 15:42:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:42:16 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <5059A01C.2080706@Media-Brokers.com> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> Message-ID: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> On 19.9.2012, at 13.36, Charles Marcus wrote: > On 2012-09-19 5:14 AM, Timo Sirainen wrote: >> It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). > > Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. Dovecot has also some additional restrictions on characters (mainly control chars), but they are only enforced when creating new mailboxes. Opening existing mailboxes with control chars is allowed. From tss at iki.fi Wed Sep 19 15:47:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:47:28 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5059B774.6060808@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> Message-ID: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> On 19.9.2012, at 15.15, Cibest ML wrote: > There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). No idea. INBOX is treated case-insensitively by IMAP protocol. > I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. From tss at iki.fi Wed Sep 19 15:48:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:48:27 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059A469.6060604@skye.it> References: <5059A469.6060604@skye.it> Message-ID: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> On 19.9.2012, at 13.54, Alessio Cecchi wrote: > LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:52:41 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:52:41 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <20120919125241.GF2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:17:50 PM GMT, Timo Sirainen wrote: > > On 18.9.2012, at 14.40, Jost Krieger wrote: > ... > > The idea is to not eat up the users storage with spam, so that spam > > stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Thank you! > Now, the next problem is that if LDA can't save the message to UCE-TMP > because of quota failure it saves it to INBOX. I'm not really sure how > that should be fixed, since generally it's a good idea to do it.. Oops, that's not what I want! How about another command line parameter for lda to suppress the emergency save? We know exactly when we are trying to save probable spam. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:53:39 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:53:39 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5058B39D.5030501@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> Message-ID: <20120919125339.GG2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > some allready existing way is i.e > with milter and postfix tagged spam get in hold queue for i.e human > admin investigate, or some quarantaine with amavis Human admin is not a good idea if you have 50000 users :-) Quarantine's a possibility, but that's taking it out of "Mail" and adds complexity. > for sure there are benefits getting dovecot involved in spam tagging > but i would leave spam tagging to the mailserver in first way and > deliver it to Junk folder by global sieve rule with giving more quota or > ignoring Quota to Junk folder, but i see this isnt the same what you > might wanna goal That's just what we want to do, *but* although it's in the Wiki I don't think it a good idea to *raise* the limit for the junk folder if you *deliver* to it. It's somewhat ok if you move mail there and someone picks it up (we do that with a different folder). But on delivery it means at some point you can receive spam but no real messages. Not nice. "ignore" may work if it does what I think (never even count bytes and messages in this folder) but it is open to abuse ... Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From amateo at um.es Wed Sep 19 16:00:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 15:00:45 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50598D49.8060507@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <50598D49.8060507@um.es> Message-ID: <5059C1FD.1040702@um.es> (I forgot the attach) El 19/09/12 11:15, Angel L. Mateo escribi?: > El 18/09/12 18:31, Timo Sirainen escribi?: >> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >> >>>>> So I'm running this command. Whenever I run it, I get the >>>>> message that 3 (sometimes, is 4) entries are removed, but user >>>>> information isn't really reloaded and I doubt it is really removed >>>>> from cache (I have the user in a passwd-file and information used >>>>> by imap processes is still the old one, no the new one, changed >>>>> before the flush) >>>> >>>> Works in my tests. >>>> >>> Is this cache the same than the user information cache? >> >> Yes. >> >>> The parameter of the user I want to change is his quota, so I >>> have modified quota value in my ldap diretory, then I run: >>> >>> doveadm auth cache flush >> >> What is your doveconf -n output and the dovecot-ldap.conf contents? Is >> with or without @domain? Also try this: >> >> doveadm auth cache flush foo # make sure it isn't there >> doveadm user foo >> doveadm auth cache flush foo >> >> Does the second flush return 1 or 0 entries? If 0, then there's a >> problem. If 1, then it really should have worked. >> >> You could try also if disabling userdb prefetch makes any difference. >> And if you still have multiple userdb try with only one. >> > > I have made the test in my test server (it has no real activity). > In this server, user entry is refreshed correctly. But the same test in > my production servers fails. > > I have checked (in the production one) that the second flush delete > entries (in fact, 2, not 1): > > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > > but quota information is not reloaded from ldap server. > > I have also checked my ldap server, and dovecot is not performing > any search operation for the user after flushing him from the cache. > > I have attached my doveconf -n. In my ldap configuration I have: > user_attrs = > irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule > > pass_attrs = > irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule > > > and I have check that my test and production server has the same > configuration (but client_limit and number of processes). > > Any idea? > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Wed Sep 19 16:03:58 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:03:58 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> Message-ID: <5059C2BE.7050006@skye.it> Il 19/09/2012 14:48, Timo Sirainen ha scritto: > On 19.9.2012, at 13.54, Alessio Cecchi wrote: > >> LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. > gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html > Obvious ... :-) Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi10 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) is enough? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Wed Sep 19 16:07:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:07:31 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C2BE.7050006@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> Message-ID: <5059C393.5050209@skye.it> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: > Il 19/09/2012 14:48, Timo Sirainen ha scritto: >> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >> >>> LDA is configured and works fine but the problem is when the first >>> message arrive "dovecot-lda" return a "Segmentation fault", the >>> message is written to the user's Mailbox but the message remains, >>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>> the second attempt is delivered fine. >> gdb backtrace would be very helpful in figuring out the problem: >> http://dovecot.org/bugreport.html >> > > Obvious ... :-) > > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. > Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. > Loaded symbols for > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so > Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver > -d cecchi10 at qboxdns.it'. > Program terminated with signal 11, Segmentation fault. > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > 221 if (dict->dict == NULL) > > is enough? This is the full bt: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = 0x0, v_modifiable = 0x0} ids = 0x1928658 i = dest = ret = -883075307 #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x1944240) at acl-backend-vfile-acllist.c:297 auser = 0x1949a08 iter = 0x0 acllist_path = 0x1928658 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" ret = ns = 0x1943e50 output = 0x0 st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1348059559, tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x1928210 file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x1928610 "Sent" origin = 0x194d178 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh (backend=0x1944240) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x19444e0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1348059559, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 26515976 #6 0x00007f2fc9fc125e in acl_backend_get_default_rights (backend=0x1944240, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x1944240 acl_mask = 0x1 ns = 0x1943e50 update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = 0x7f2fcbf2ba88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ctx = 0x1946b20 pool = i = inboxcase = #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1348059559) at mailbox-list-iter.c:58 patterns = {0x7f2fc9db76dc "*", 0x0} #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:73 ctx = 0x7f2fcb5beef3 info = #12 quota_count (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff362e0038) at quota-dict.c:113 dt = bytes = 0 count = 0 #14 0x00007f2fc9db395a in dict_quota_update_callback ( ret=, context=0x1949a08) at quota-dict.c:178 value = 1 #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, line_r=) at dict-client.c:265 ctx = 0x1984190 #16 client_dict_read_one_line (dict=0x194c7f0, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f2fcb592565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x194c7f0 line = 0x0 ret = #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x1944950 #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x194c5c0 #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 quota = 0x1944720 i = 2 #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) at quota-storage.c:412 quser = 0x1949a30 quota_set = 0x1946f30 #22 0x00007f2fcb88188e in mail_user_unref (_user=) at mail-user.c:153 user = 0x19483c0 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x192e050, tried_default_save = true, saved_mail = true, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = user = 0x192d3c2 "cecchi10 at qboxdns.it" errstr = 0x0 path = 0x7fff362e0368 "\351\a" storage_service = 0x192f3a0 service_user = 0x192fd48 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x1943f30 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ret = c = error = MAIL_ERROR_NONE (gdb) Sorry -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From robert at schetterer.org Wed Sep 19 17:03:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:03:45 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125339.GG2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> Message-ID: <5059D0C1.5050202@schetterer.org> Am 19.09.2012 14:53, schrieb Jost Krieger: > On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > >> some allready existing way is i.e >> with milter and postfix tagged spam get in hold queue for i.e human >> admin investigate, or some quarantaine with amavis > > Human admin is not a good idea if you have 50000 users :-) > Quarantine's a possibility, but that's taking it out of "Mail" and adds > complexity. as ever ,this is different in different places specially Quarantine is wide used > >> for sure there are benefits getting dovecot involved in spam tagging >> but i would leave spam tagging to the mailserver in first way and >> deliver it to Junk folder by global sieve rule with giving more quota or >> ignoring Quota to Junk folder, but i see this isnt the same what you >> might wanna goal > > That's just what we want to do, *but* although it's in the Wiki I don't > think it a good idea to *raise* the limit for the junk folder if you > *deliver* to it. It's somewhat ok if you move mail there and someone > picks it up (we do that with a different folder). But on delivery it > means at some point you can receive spam but no real messages. Not nice. > > "ignore" may work if it does what I think (never even count bytes and > messages in this folder) but it is open to abuse ... there is no good or evil to this case, its a design question which may change in different places,, the abuse case might be acceptable, as the Junk folder i.e is imap special used autoconfigured and has auto empty feature, i give 50 mb plus quota on Junk folder and delete mail older then 3 month there > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 19 17:25:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:25:05 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <5059D5C1.500@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig > Hi Timo, i see a few rare errors with lucene and/or indexer since update /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out lucene-indexes) failed: Directory not empty Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) failed: Directory not empty lucene-indexes: IndexReader::open() failed (#1): File does not exist Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual method called Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called without an active exception Error: indexer failed to index mailbox INBOX should i care -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Wed Sep 19 17:25:30 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 16:25:30 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5059D0C1.5050202@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> Message-ID: <20120919142529.GI2010@ruhr-uni-bochum.de> On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > as ever ,this is different in different places > specially Quarantine is wide used Of course, it's often useful. > there is no good or evil to this case, its a design question which may > change in different places,, the abuse case might be acceptable, as the > Junk folder i.e is imap special used autoconfigured > and has auto empty feature, i give 50 mb plus quota on Junk folder > and delete mail older then 3 month there The abuse may be tolerable, but I just don't get how your +50M works. For me it would mean that someone with a large Junk folder just wouldn't get an more mail except something delivered straight to Junk. And if you don't deliver to Junk but the clients move stuff there, there's still a problem. I'm justing raising the limit for folders that clients use for deleting (like Trash or whatever it is called). Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From robert at schetterer.org Wed Sep 19 17:44:42 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:44:42 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919142529.GI2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> <20120919142529.GI2010@ruhr-uni-bochum.de> Message-ID: <5059DA5A.1070908@schetterer.org> Am 19.09.2012 16:25, schrieb Jost Krieger: > On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > >> as ever ,this is different in different places >> specially Quarantine is wide used > > Of course, it's often useful. > >> there is no good or evil to this case, its a design question which may >> change in different places,, the abuse case might be acceptable, as the >> Junk folder i.e is imap special used autoconfigured >> and has auto empty feature, i give 50 mb plus quota on Junk folder >> and delete mail older then 3 month there > > The abuse may be tolerable, but I just don't get how your +50M works. the situation isnt really comparable, the plus is more configured for copy by human to i.e Junk folder, or i.e a client filter does this auto it makes most sense with trash, so user should mostly able to copy to trash, even if they have got full quota, this should avoid some errors in clients http://wiki2.dovecot.org/Quota/Configuration ... quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = SPAM:ignore ... > For me it would mean that someone with a large Junk folder just wouldn't > get an more mail except something delivered straight to Junk. And if you > don't deliver to Junk but the clients move stuff there, with auto loosing mail after some time in my setup which is the anounced policy, Trash and Junk are "autodeleted" folders no mail there ,rests forever *g there's still a > problem. I'm justing raising the limit for folders that clients use for > deleting (like Trash or whatever it is called). as i said its a design question, your policy is ok too no flame ,having your idea as alternative is always fine to have > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From interfasys at gmail.com Wed Sep 19 17:55:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 15:55:33 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <5059DCE5.8030903@gmail.com> > Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. You are correct, it had not been compiled in a while. Here is a new backtrace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at user.c:96 _module_tmp = (void *) 0x10a84d18 asu = (struct antispam_user *) 0x10a84d18 tmp = Variable "tmp" is not available. Cheers, Olivier From tss at iki.fi Wed Sep 19 17:59:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 17:59:38 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5059DCE5.8030903@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> Message-ID: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > You are correct, it had not been compiled in a while. Here is a new > backtrace: .. > #6 0x00000000105e7a6a in buffer_write () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #7 0x00000000105ce9f9 in array_idx_set_i () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > user.c:96 > _module_tmp = (void *) 0x10a84d18 > asu = (struct antispam_user *) 0x10a84d18 > tmp = Variable "tmp" is not available. This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? From tss at iki.fi Wed Sep 19 18:02:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:02:07 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> Message-ID: <1348066927.13571.69.camel@hurina> On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: > On 19.9.2012, at 17.55, interfaSys s?rl wrote: > > >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > > > You are correct, it had not been compiled in a while. Here is a new > > backtrace: > .. > > #6 0x00000000105e7a6a in buffer_write () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #7 0x00000000105ce9f9 in array_idx_set_i () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > > user.c:96 > > _module_tmp = (void *) 0x10a84d18 > > asu = (struct antispam_user *) 0x10a84d18 > > tmp = Variable "tmp" is not available. > > This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? Attached a helpful antispam patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 444 bytes Desc: not available URL: From interfasys at gmail.com Wed Sep 19 18:12:01 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 16:12:01 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: <5059E0C1.9070906@gmail.com> You are once again correct. The update script had an error and thus the plugin was not recompiled for 2.1.10... Thank you! Olivier On 19/09/2012 16:02, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >>>> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. >>> >>> You are correct, it had not been compiled in a while. Here is a new >>> backtrace: >> .. >>> #6 0x00000000105e7a6a in buffer_write () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #7 0x00000000105ce9f9 in array_idx_set_i () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >>> user.c:96 >>> _module_tmp = (void *) 0x10a84d18 >>> asu = (struct antispam_user *) 0x10a84d18 >>> tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? > > Attached a helpful antispam patch. > From tss at iki.fi Wed Sep 19 18:16:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:16:58 +0300 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <5059D5C1.500@schetterer.org> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> Message-ID: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> On 19.9.2012, at 17.25, Robert Schetterer wrote: > Am 18.09.2012 20:52, schrieb Timo Sirainen: >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >> > > Hi Timo, i see a few rare errors with lucene and/or indexer > since update Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out > lucene-indexes) failed: Directory not empty > > Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) > failed: Directory not empty > > lucene-indexes: IndexReader::open() failed (#1): File does not exist The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual > method called > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called > without an active exception > > Error: indexer failed to index mailbox INBOX > > should i care > -- > Best Regards > MfG Robert Schetterer > From eugene at raptor.kiev.ua Wed Sep 19 19:13:13 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Wed, 19 Sep 2012 19:13:13 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On Wed, 19 Sep 2012 18:02:07 +0300, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >> >> Did you recompile antispam plugin? Is it the latest antispam? This >> line doesn't seem to match the current code. >> > >> > You are correct, it had not been compiled in a while. Here is a new >> > backtrace: >> .. >> > #6 0x00000000105e7a6a in buffer_write () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #7 0x00000000105ce9f9 in array_idx_set_i () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >> > user.c:96 >> > _module_tmp = (void *) 0x10a84d18 >> > asu = (struct antispam_user *) 0x10a84d18 >> > tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile >> antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is >> also really from v2.1.10? > > Attached a helpful antispam patch. Erm... Am I missing something? http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c line 26, or actually line 32 given http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jcblanco at fi.upm.es Wed Sep 19 19:22:29 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Wed, 19 Sep 2012 18:22:29 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058C22F.1070704@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> Message-ID: <5059F145.1030803@fi.upm.es> On 18/09/2012 20:49, Stephan Bosch wrote: > On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the >> new pigeonhole version, however I've got the follwing errors when >> compiling: >> >> It seems that the error is related to the ld.gold fix, If I revert the >> c52a0c561311 patch the error is not produced > > Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, > but I forgot the Dovecot v2.1 tree. Fixed now: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > This only happens when you compile against installed Dovecot headers, > people using --with-dovecot= (like me) would not have seen this. > > I'll push out a new release once 2.1.10 comes out. I'm being told that > is imminent. Sorry, but using the new 0.3.3 version I'm getting the same error and I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in configure, not the installed headers. If needed I can send you the configure invocation command. Regards Juan C. Blanco > > Regards, > > Stephan. > > From tss at iki.fi Wed Sep 19 19:24:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 19:24:22 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On 19.9.2012, at 19.13, Eugene Paskevich wrote: >>> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? >> >> Attached a helpful antispam patch. > > Erm... Am I missing something? > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c > line 26, or actually line 32 given > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 Oh, didn't notice it there below :) Wonder why it then didn't complain about version mismatch. From alec at alec.pl Wed Sep 19 20:10:16 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 19 Sep 2012 19:10:16 +0200 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> Message-ID: <5059FC78.7090100@alec.pl> On 09/19/2012 02:42 PM, Timo Sirainen wrote: > mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. I'm confused. I have no problems with creating and using a folder with utf8 name containing '&' character. So, what problem do you describe? doveadm is working with utf8 right? So, why it can't delete such folder. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From florob at babelmonkeys.de Wed Sep 19 20:43:09 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 19:43:09 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> Message-ID: <505A042D.5090705@babelmonkeys.de> Am 11.09.2012 20:21, schrieb Timo Sirainen: > On 11.9.2012, at 21.07, Florian Zeitz wrote: > >>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>> >> Well, either that, or we could use a union of all known context structs >> there. > > That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. > >> Or we could use i_malloc() and i_free() under the assumption hmac_init() >> + hmac_final() calls are always matched. > > Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. > Here is an updated version of the patch. It uses the variant defining the largest allowed content size. I choose 1024 bit (128 Byte), since that is what most of the SHA-3 candidates use, so it's hopefully somewhat future proof. It's also rebased to the current head and switches the hmac uses added since the last version over to the changed API. It also fixes an "interesting" compiler warning I got with clang, which complained about the fact that struct hmac_context* was (re)declared in the prototype of hmac_md5_get_cram_context() and therefore separate from the one declared in hmac.h. Regards, Florian -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 0aa0a656176c9ca1c3f187999cd4725bf52665e5 # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE (1024 / 8) + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From interfasys at gmail.com Wed Sep 19 20:52:20 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 18:52:20 +0100 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <505A0654.3010501@gmail.com> I see the same crashes on FreeBSD. Turning off ACL makes the crashes disappear. I'm getting a different backtrace though and it doesn't seem to be related to ACL: Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libssl.so.8...done. Loaded symbols for /usr/local/lib/libssl.so.8 Reading symbols from /usr/local/lib/libcrypto.so.8...done. Loaded symbols for /usr/local/lib/libcrypto.so.8 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /lib/libthr.so.3...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/lib/libz.so...done. Loaded symbols for /usr/lib/libz.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 143 return (_c < 0 || _c >= __mb_sb_limit) ? _c : [New Thread 10e041c0 (LWP 103332/initial thread)] (gdb) bt full #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 table = (struct hash_table *) 0x10e29700 #1 0x0000000010478cf1 in settings_parser_deinit (_ctx=0x10e29778) at array.h:197 ctx = (struct setting_parser_context *) 0x0 #2 0x00000000104ac4ea in master_service_settings_cache_deinit (_cache=0x1045da08) at master-service-settings-cache.c:81 cache = (struct master_service_settings_cache *) 0x10e2e040 entry = (struct settings_entry *) 0x10e29740 next = (struct settings_entry *) 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x000000001044d1df in login_settings_deinit () at login-settings.c:129 No locals. #4 0x000000001044dd7e in main_deinit () at main.c:62 No locals. #5 0x000000001044df5c in login_binary_run (binary=0x4051c0, argc=2, argv=0x7fffffffebd8) at main.c:62 service_flags = 72 set_pool = 0x10e2d020 allow_core_dumps = true login_socket = 0x10457491 "login" c = -1 #6 0x00000000004036ee in main (argc=2, argv=0x7fffffffebd8) at client.c:40 No locals. Cheers, Olivier From stephan at rename-it.nl Wed Sep 19 21:07:12 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 19 Sep 2012 20:07:12 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5059F145.1030803@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> Message-ID: <505A09D0.6040607@rename-it.nl> On 9/19/2012 6:22 PM, Juan C. Blanco wrote: > On 18/09/2012 20:49, Stephan Bosch wrote: >> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>> It seems that the error is related to the ld.gold fix, If I revert the >>> c52a0c561311 patch the error is not produced >> >> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >> but I forgot the Dovecot v2.1 tree. Fixed now: >> >> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c D'oh, this is the v2.2 change. The URL I meant is this one: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >> >> This only happens when you compile against installed Dovecot headers, >> people using --with-dovecot= (like me) would not have seen this. >> >> I'll push out a new release once 2.1.10 comes out. I'm being told that >> is imminent. > > Sorry, but using the new 0.3.3 version I'm getting the same error and > I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in > configure, not the installed headers. If needed I can send you the > configure invocation command. Yes, that could be helpful. Regards, Stephan. From florob at babelmonkeys.de Wed Sep 19 21:42:50 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 20:42:50 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <505A042D.5090705@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> <505A042D.5090705@babelmonkeys.de> Message-ID: <505A122A.3070307@babelmonkeys.de> Am 19.09.2012 19:43, schrieb Florian Zeitz: > Am 11.09.2012 20:21, schrieb Timo Sirainen: >> On 11.9.2012, at 21.07, Florian Zeitz wrote: >> >>>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>>> >>> Well, either that, or we could use a union of all known context structs >>> there. >> >> That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. >> >>> Or we could use i_malloc() and i_free() under the assumption hmac_init() >>> + hmac_final() calls are always matched. >> >> Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. >> > Here is an updated version of the patch. > It uses the variant defining the largest allowed content size. > I choose 1024 bit (128 Byte), since that is what most of the SHA-3 > candidates use, so it's hopefully somewhat future proof. Forget I said that. Unfortunately the MD5 context is already bigger (the SHA-1 one isn't). So 256 Byte it is I guess. Sorry for the lack of thorough testing. I hope I didn't miss another hash function. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 5e6802858a5bb3a9b278f1cbdabe5c0ffb15248d # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE 256 + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From c at roessner-network-solutions.com Wed Sep 19 22:48:17 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 21:48:17 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: Hi, > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 unfortunately z-Push is much too buggy here. So I dropped it in favor of standard IMAP. Best wishes -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From ben at indietorrent.org Thu Sep 20 00:41:38 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 17:41:38 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504F4D5D.8040207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> Message-ID: <505A3C12.7050207@indietorrent.org> On 9/11/2012 10:40 AM, Ben Johnson wrote: > You nailed it, Ben. Thanks! > > I downloaded/installed the dovecot-dev package from the same maintainer > and was able to build Antispam without issue. > > The hang-up now seems to be with my configuration. > > I'm seeing the following error in my email client whenever I attempt to > move an email message from, say, Inbox to Junk folder: > > [SERVERBUG] BUG Unknown internal error. > > Nothing at all appears in syslog when this happens. > > I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam > , to the letter, I believe. > > The relevant portions of my configuration are as follows: > > --------------------- > protocol imap { > mail_plugins = quota imap_quota antispam > } > > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > > antispam_backend = MAILTRAIN > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > antispam_spam_pattern_ignorecase = spam;junk > } > --------------------- > > The permissions and execute bit are set in accordance with other > executables in the same directory: > > # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" > -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh > > I would provide more log content, but the debug options seem to be gone > in the forked, >= 2.0 version of this plug-in. > > Any ideas are very much appreciated. > > Thanks, > > -Ben Given that nobody shared any insight here, I decided to stand-up a completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. While the error is different this time around, I am still unable to get any variant of the dovecot-antispam plug-in working. When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and I use the above configuration, my syslog fills-up with the following: dovecot: imap-login: Login: user=, method=PLAIN, rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS dovecot: master: Error: service(imap): child 2879 returned error 3 Surprisingly, I can find only two other mentions of "error 3" anywhere on the Internet. Those mentions were from another user to this mailing list: http://www.dovecot.org/list/dovecot/2012-August/067787.html http://www.dovecot.org/list/dovecot/2012-August/067814.html The lack of "error 3" ubiquity leads me to believe that I must be doing something stupid. There's no question that this behavior is related to the antispam-plugin, because when I commented-out the relevant lines in Dovecot's configuration file, the error does not occur. I have followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the result. The only instance in which I diverged from the Wiki article is that I used the Ubuntu-provided dovecot-antispam package (instead of compiling the plug-in from source). I am trying to use the "mailtrain" back-end, per the above-cited Wiki article, to call SpamAssassin's "sa-learn" utility for real-time learning. I am not using the SpamAssassin daemon (spamd). I'm about to throw in the towel on this plug-in, so any help would be greatly appreciated. Thank you, -Ben From tom at whyscream.net Thu Sep 20 01:19:07 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 20 Sep 2012 00:19:07 +0200 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505A44DB.6020200@whyscream.net> On 19/09/12 23:41, Ben Johnson wrote: > > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben There are 2 versions of the dovecot-antispam package. The original version was created by Johannes Berg, then died almost a silent death, and was forked by Eugene Paskevitch. Later Johannes revived his original version somewhat. The ubuntu package is based on the 'revived Johannes fork' [1], while the documentation on the dovecot wiki describes the 'Eugene fork'. [2] [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright [2] http://wiki2.dovecot.org/Plugins/Antispam Thus you'll need to choose which plugin you want, then follow the complete procedure for that plugin only. They both can do what you want, but building and configuration are slightly different. It would be nice if Johannes and Eugene would either agree on a single version of the plugin, or rename one of the forks so they don't get mixed up that easily. I'm not sure if that has been discussed (on- or off-list) in the past. FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to upgrade to 2.1 yet, so no experience there. -- Tom From ben at indietorrent.org Thu Sep 20 02:17:43 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 19:17:43 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A44DB.6020200@whyscream.net> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> Message-ID: <505A5297.8050808@indietorrent.org> On 9/19/2012 6:19 PM, Tom Hendrikx wrote: > On 19/09/12 23:41, Ben Johnson wrote: >> >> >> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>> You nailed it, Ben. Thanks! >>> >>> I downloaded/installed the dovecot-dev package from the same maintainer >>> and was able to build Antispam without issue. >>> >>> The hang-up now seems to be with my configuration. >>> >>> I'm seeing the following error in my email client whenever I attempt to >>> move an email message from, say, Inbox to Junk folder: >>> >>> [SERVERBUG] BUG Unknown internal error. >>> >>> Nothing at all appears in syslog when this happens. >>> >>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>> , to the letter, I believe. >>> >>> The relevant portions of my configuration are as follows: >>> >>> --------------------- >>> protocol imap { >>> mail_plugins = quota imap_quota antispam >>> } >>> >>> plugin { >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> sieve = /var/vmail/%d/%n/.sieve >>> >>> antispam_backend = MAILTRAIN >>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>> antispam_mail_spam = --spam >>> antispam_mail_notspam = --ham >>> antispam_spam_pattern_ignorecase = spam;junk >>> } >>> --------------------- >>> >>> The permissions and execute bit are set in accordance with other >>> executables in the same directory: >>> >>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>> >>> I would provide more log content, but the debug options seem to be gone >>> in the forked, >= 2.0 version of this plug-in. >>> >>> Any ideas are very much appreciated. >>> >>> Thanks, >>> >>> -Ben >> >> Given that nobody shared any insight here, I decided to stand-up a >> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >> >> While the error is different this time around, I am still unable to get >> any variant of the dovecot-antispam plug-in working. >> >> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >> I use the above configuration, my syslog fills-up with the following: >> >> dovecot: imap-login: Login: user=, method=PLAIN, >> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >> dovecot: master: Error: service(imap): child 2879 returned error 3 >> >> Surprisingly, I can find only two other mentions of "error 3" anywhere >> on the Internet. Those mentions were from another user to this mailing list: >> >> http://www.dovecot.org/list/dovecot/2012-August/067787.html >> http://www.dovecot.org/list/dovecot/2012-August/067814.html >> >> The lack of "error 3" ubiquity leads me to believe that I must be doing >> something stupid. >> >> There's no question that this behavior is related to the >> antispam-plugin, because when I commented-out the relevant lines in >> Dovecot's configuration file, the error does not occur. >> >> I have followed the instructions at >> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >> result. >> >> The only instance in which I diverged from the Wiki article is that I >> used the Ubuntu-provided dovecot-antispam package (instead of compiling >> the plug-in from source). >> >> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >> >> I am not using the SpamAssassin daemon (spamd). >> >> I'm about to throw in the towel on this plug-in, so any help would be >> greatly appreciated. >> >> Thank you, >> >> -Ben > > There are 2 versions of the dovecot-antispam package. The original > version was created by Johannes Berg, then died almost a silent death, > and was forked by Eugene Paskevitch. Later Johannes revived his original > version somewhat. > > The ubuntu package is based on the 'revived Johannes fork' [1], while > the documentation on the dovecot wiki describes the 'Eugene fork'. [2] > > [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright > [2] http://wiki2.dovecot.org/Plugins/Antispam > > Thus you'll need to choose which plugin you want, then follow the > complete procedure for that plugin only. They both can do what you want, > but building and configuration are slightly different. > > It would be nice if Johannes and Eugene would either agree on a single > version of the plugin, or rename one of the forks so they don't get > mixed up that easily. I'm not sure if that has been discussed (on- or > off-list) in the past. > > FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the > 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to > upgrade to 2.1 yet, so no experience there. > > -- > Tom > Thank you for your timely response, Tom. Much appreciated. Your remarks clear-up a lot of the confusion for me. I didn't realize that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did I realize that this is the version that is packaged with Ubuntu 12.04. Given that I'm more-or-less forced to work with Ubuntu LTS, I would prefer to use the OS's package management system wherever possible. I suppose, by extension, that I would prefer to use the "revived Johannes fork". I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but am unable to get either to work the way I want (mailtrain back-end with real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki article). Shouldn't the basic tenets of each version of the plug-in apply? Where should I start, in the way of seeking support? Basically, I intend to use the Johannes version, but I want to configure it in the manner described in the Eugene version Wiki article. I've already described (rather exhaustively) what happens when I attempt to use the Johannes version and adapt the Eugene version instructions. I'm curious to know what method you've used over the years with such glowing success. Which back-end did you use with Dovecot 1/Johannes? How about with Dovecot 2/Eugene? Thanks again, -Ben From selcuk.yazar at gmail.com Thu Sep 20 09:21:21 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:21:21 +0300 Subject: [Dovecot] Prevent Download messages from server Message-ID: Hi, can we prevent download messages from server user by user ? sme common used mail's message must be remain at the server, but sometimes we download them ? thanks in advance -- Sel?uk YAZAR From robert at schetterer.org Thu Sep 20 09:28:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:28:37 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AB795.3000106@schetterer.org> Am 20.09.2012 08:21, schrieb Selcuk Yazar: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > please clear , what you mean with 1 : with download messages 2 : user by user 3 : sme 4 : mail's message must be remain at the server 5 : but sometimes we download them show your dovecot conf and problem logs -- Best Regards MfG Robert Schetterer From selcuk.yazar at gmail.com Thu Sep 20 09:31:19 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:31:19 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AB795.3000106@schetterer.org> References: <505AB795.3000106@schetterer.org> Message-ID: Hi, we have no problem, just i want to learn how can i do that. i think it's clear . sel?uk On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > Am 20.09.2012 08:21, schrieb Selcuk Yazar: > > Hi, > > > > can we prevent download messages from server user by user ? sme common > used > > mail's message must be remain at the server, but sometimes we download > them > > ? > > > > thanks in advance > > > > please clear , what you mean with > > 1 : with download messages > 2 : user by user > 3 : sme > 4 : mail's message must be remain at the server > 5 : but sometimes we download them > > show your dovecot conf and problem logs > > -- > Best Regards > MfG Robert Schetterer > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From robert at schetterer.org Thu Sep 20 09:38:48 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:38:48 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AB795.3000106@schetterer.org> Message-ID: <505AB9F8.6060603@schetterer.org> Am 20.09.2012 08:31, schrieb Selcuk Yazar: > Hi, > > we have no problem, just i want to learn how can i do that. i think it's > clear . > > > sel?uk ok ,so the right answer is, unless you give more details stop dovecot , nobody will able to connect to it > > On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > >> Am 20.09.2012 08:21, schrieb Selcuk Yazar: >>> Hi, >>> >>> can we prevent download messages from server user by user ? sme common >> used >>> mail's message must be remain at the server, but sometimes we download >> them >>> ? >>> >>> thanks in advance >>> >> >> please clear , what you mean with >> >> 1 : with download messages >> 2 : user by user >> 3 : sme >> 4 : mail's message must be remain at the server >> 5 : but sometimes we download them >> >> show your dovecot conf and problem logs >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > -- Best Regards MfG Robert Schetterer From alessio at skye.it Thu Sep 20 09:45:03 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:45:03 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ABB6F.2010606@skye.it> Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > If your users can log and read messages can obviously download it. If you use IMAP you can set ACL and enable read only for the folders: http://wiki2.dovecot.org/ACL -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Thu Sep 20 09:51:14 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:51:14 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <505A0654.3010501@gmail.com> References: <505A0654.3010501@gmail.com> Message-ID: <505ABCE2.30802@skye.it> Il 19/09/2012 19:52, interfaSys s?rl ha scritto: > I see the same crashes on FreeBSD. Turning off ACL makes the crashes > disappear. > > I'm getting a different backtrace though and it doesn't seem to be > related to ACL: > > Core was generated by `imap-login'. > Thanks for your report. My backtrace was genereted from "dovecot-lda", but in both cases disable ACL "solve" the problem. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From selcuk.yazar at gmail.com Thu Sep 20 09:53:28 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:53:28 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ABB6F.2010606@skye.it> References: <505ABB6F.2010606@skye.it> Message-ID: Thank you, i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think it's diffucult to evaluate in POP3. also we are using openLDAP backend, if i change folder permissions , does it work ? sel?uk On Thu, Sep 20, 2012 at 9:45 AM, Alessio Cecchi wrote: > Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > >> Hi, >> >> can we prevent download messages from server user by user ? sme common >> used >> mail's message must be remain at the server, but sometimes we download >> them >> ? >> >> thanks in advance >> >> > If your users can log and read messages can obviously download it. > > If you use IMAP you can set ACL and enable read only for the folders: > > http://wiki2.dovecot.org/ACL > > -- > Alessio Cecchi is: > @ ILS -> http://www.linux.it/~alessice/ > on LinkedIn -> http://www.linkedin.com/in/**alessice > Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ > @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From odhiambo at gmail.com Thu Sep 20 10:13:08 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 20 Sep 2012 10:13:08 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: On Thu, Sep 20, 2012 at 9:53 AM, Selcuk Yazar wrote: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Changing folder permissions to what? I suppose you understand what you mean by changing folder permissions! If you make them read-only (no write) for the users group, as that is what you could be thinking, you'll probably also affect the ability of the MTA to write to them (deliver mail). Please explain your situation holistically so that we can understand and give suggestions... -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. From alessio at skye.it Thu Sep 20 10:19:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 09:19:31 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: <505AC383.2010107@skye.it> Il 20/09/2012 08:53, Selcuk Yazar ha scritto: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Change folder permissions is a bad idea, for example if you use Maildir/ as mailbox format you can't remove +w, messages should be moved from new/ to cur/, add flags to name and so on. I do not think it's possible to do what you ask. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From apm at one.com Thu Sep 20 10:24:34 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 20 Sep 2012 09:24:34 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AC4B2.2040900@one.com> > we have no problem, just i want to learn how can i do that. i think > it's clear . Well... I'm pretty sure most others don't. But anyway. As in ALL Internet protocols (IMAP being no exception), letting the client read data on the server requires it to download the data. Preventing download will prevent reading the mail. Period. So if you're fine with that and just want to learn how to do it, then just disable the account in the user database. /Peter From selcuk.yazar at gmail.com Thu Sep 20 10:28:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:28:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: Alessio and Peter thank you again. maybe it can be future request. :) selcuk. On Thu, Sep 20, 2012 at 10:24 AM, Peter Mogensen wrote: > > we have no problem, just i want to learn how can i do that. i think > > it's clear . > > Well... I'm pretty sure most others don't. > > But anyway. As in ALL Internet protocols (IMAP being no exception), > letting the client read data on the server requires it to download the data. > Preventing download will prevent reading the mail. Period. > > So if you're fine with that and just want to learn how to do it, then just > disable the account in the user database. > > /Peter > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From jc at info-systems.de Thu Sep 20 10:51:41 2012 From: jc at info-systems.de (Jakob Curdes) Date: Thu, 20 Sep 2012 09:51:41 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AC4B2.2040900@one.com> Message-ID: <505ACB0D.7020600@info-systems.de> Am 20.09.2012 09:28, schrieb Selcuk Yazar: > Alessio and Peter thank you again. maybe it can be future request. :) We are trying to tell you that it is impossible to write a mail _client_ that does not download (i.e. transfer) the mail that it shows to the user. This is why it makes no sense to prevent the download on the server side, and so there will never be such a function. Preventing the download (the transfer) of the mail fro mthe server to the client just means that the user *cannot read it*. Another question is whether there are mail _clients_ which do not offer a possibility to *save* the read mail (which has already been "downloaded" for reading it). But that question is not really related to the dovecot mail server. If you want to get users to read mails, but not download them, I would suggest you deny the direct access to the mail server and use a browser-based application that shows the entire mail to the user, maybe as a graphical representation. Or you use some sort of terminal software that connects to a host where the mail _clients_ itself are installed, just letting the user view the data but nor copy it, e.g. the VNC protocol. Hope this helps, Jakob Curdes From selcuk.yazar at gmail.com Thu Sep 20 10:54:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:54:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ACB0D.7020600@info-systems.de> References: <505AC4B2.2040900@one.com> <505ACB0D.7020600@info-systems.de> Message-ID: why are you trying ??? this is brain storming On Thu, Sep 20, 2012 at 10:51 AM, Jakob Curdes wrote: > Am 20.09.2012 09:28, schrieb Selcuk Yazar: > > Alessio and Peter thank you again. maybe it can be future request. :) > > We are trying to tell you that it is impossible to write a mail *client*that does not download (i.e. transfer) the mail that it shows to the user. > This is why it makes no sense to prevent the download on the server side, > and so there will never be such a function. > Preventing the download (the transfer) of the mail fro mthe server to the > client just means that the user *cannot read it*. > > Another question is whether there are mail *clients* which do not offer a > possibility to *save* the read mail (which has already been "downloaded" > for reading it). But that question is not really related to the dovecot > mail server. > > If you want to get users to read mails, but not download them, I would > suggest you deny the direct access to the mail server and use a > browser-based application that shows the entire mail to the user, maybe as > a graphical representation. Or you use some sort of terminal software > that connects to a host where the mail *clients* itself are installed, > just letting the user view the data but nor copy it, e.g. the VNC protocol. > > Hope this helps, > Jakob Curdes > > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From Bill at knoxvillechristian.org Thu Sep 20 11:32:54 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 20 Sep 2012 04:32:54 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505AD4B6.1020109@knoxvillechristian.org> On 9/19/2012 5:41 PM, Ben Johnson wrote: > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben I've never used this plugin but let offer up my home brewed script that I call from crontab: # Process and delete .Spam and .Ham Maildirs 11 */4 * * * /usr/local/sbin/maildir.sa-learn and /usr/local/sbin/maildir.sa-learn: #!/bin/bash MAILDIR=Maildir LS=/bin/ls RM=/bin/rm SU=/bin/su FIND=/bin/find SALEARN=/usr/bin/sa-learn for i in `$LS /home`; do prefs="/home/$i/.spamassassin/user_prefs" # [ ! ."$i" == ".bill" ] && continue echo $i spamdir="/home/$i/$MAILDIR/.SystemFolders.Spam" [ ! -d "$spamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/new" $i $FIND "$spamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/cur" $i $FIND "$spamdir/cur" -type f -delete hamdir="/home/$i/$MAILDIR/.SystemFolders.Ham" [ ! -d "$hamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/new" $i $FIND "$hamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/cur" $i $FIND "$hamdir/cur" -type f -delete done With my setup each user has their own spamassassin user_prefs and bayes tokens. Bill From robert at schetterer.org Thu Sep 20 11:42:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 10:42:05 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: <505AD6DD.3020700@schetterer.org> Am 20.09.2012 09:24, schrieb Peter Mogensen: > Preventing download will prevent reading the mail famous last words *g -- Best Regards MfG Robert Schetterer From david at blue-labs.org Thu Sep 20 12:16:06 2012 From: david at blue-labs.org (David Ford) Date: Thu, 20 Sep 2012 05:16:06 -0400 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ADED6.7030204@blue-labs.org> do you mean to leave a copy of the email on the server so it can be read in multiple email clients? IMAP can do this and i think modern POP3 can. look for an account config option in your mail client to "leave mail on server". i think there is a setting in dovecot to prevent expunging of email but it has been years since i was researching this. -david On 09/20/2012 02:21 AM, Selcuk Yazar wrote: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > From robert at schetterer.org Thu Sep 20 17:51:04 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 16:51:04 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> Message-ID: <505B2D58.3010301@schetterer.org> Am 19.09.2012 17:16, schrieb Timo Sirainen: > On 19.9.2012, at 17.25, Robert Schetterer wrote: > >> Am 18.09.2012 20:52, schrieb Timo Sirainen: >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >>> >> >> Hi Timo, i see a few rare errors with lucene and/or indexer >> since update > > Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > >> /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out >> lucene-indexes) failed: Directory not empty >> >> Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) >> failed: Directory not empty >> >> lucene-indexes: IndexReader::open() failed (#1): File does not exist > > The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual >> method called >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called >> without an active exception >> >> Error: indexer failed to index mailbox INBOX >> >> should i care >> -- >> Best Regards >> MfG Robert Schetterer >> > update: the above errors didnt happen again today for one time this error comes up instead Error: Log synchronization error at seq=8,offset=28596 for /usr/local/virtual/...../...../dovecot.index: Extension header update without intro prefix -- Best Regards MfG Robert Schetterer From surreal at bsdmail.com Thu Sep 20 18:24:16 2012 From: surreal at bsdmail.com (Surreal) Date: Thu, 20 Sep 2012 23:24:16 +0800 Subject: [Dovecot] userdb missing driver Message-ID: <505B3520.10309@bsdmail.com> Good Day, I'm using dovecot 2.1.8 and this is my dovecot -n output: # 2.1.8: dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = domain.xxx auth_mechanisms = plain login auth_realms = domainone.xxx domaintwo.xxx disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver ---------------------------- This is my dovecot-sql.conf.ext connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix driver = mysql default_pass_scheme = MD5-CRYPT password_query = SELECT password,CONCAT('/mail/', maildir) AS userdb_home,\ '5000' AS userdb_uid, '5000' AS userdb_gid, allow_nets,\ concat('*:bytes=', quota) AS userdb_quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 user_query = SELECT CONCAT('/mail/', maildir) AS home, 'maildir:~/' as mail, '5000' AS uid, '5000' AS gid,\ concat('*:bytes=', quota) AS quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 #iterate_query = SELECT username AS user FROM mailbox ------------------------------ I'm sure i granted all privileges to daemonuser to postfix db. This is my first on dovecot so my knowledge is very minimal, all help possible is very much appreciated. Thanks From h.reindl at thelounge.net Thu Sep 20 18:43:55 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 20 Sep 2012 17:43:55 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3520.10309@bsdmail.com> References: <505B3520.10309@bsdmail.com> Message-ID: <505B39BB.3060308@thelounge.net> Am 20.09.2012 17:24, schrieb Surreal: > Good Day, > > I'm using dovecot 2.1.8 and this is my dovecot -n output: > > # 2.1.8: dovecot.conf > # OS: FreeBSD 9.0-STABLE i386 > > doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver > > connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix > driver = mysql no idea about BSD but on linux systems these are typically seperated packages dovecot-2.1.10-4.fc17.20120919.rh.x86_64 dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ben at indietorrent.org Thu Sep 20 18:53:12 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 11:53:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A5297.8050808@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> Message-ID: <505B3BE8.5050202@indietorrent.org> On 9/19/2012 7:17 PM, Ben Johnson wrote: > > > On 9/19/2012 6:19 PM, Tom Hendrikx wrote: >> On 19/09/12 23:41, Ben Johnson wrote: >>> >>> >>> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>>> You nailed it, Ben. Thanks! >>>> >>>> I downloaded/installed the dovecot-dev package from the same maintainer >>>> and was able to build Antispam without issue. >>>> >>>> The hang-up now seems to be with my configuration. >>>> >>>> I'm seeing the following error in my email client whenever I attempt to >>>> move an email message from, say, Inbox to Junk folder: >>>> >>>> [SERVERBUG] BUG Unknown internal error. >>>> >>>> Nothing at all appears in syslog when this happens. >>>> >>>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>>> , to the letter, I believe. >>>> >>>> The relevant portions of my configuration are as follows: >>>> >>>> --------------------- >>>> protocol imap { >>>> mail_plugins = quota imap_quota antispam >>>> } >>>> >>>> plugin { >>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>> sieve = /var/vmail/%d/%n/.sieve >>>> >>>> antispam_backend = MAILTRAIN >>>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>>> antispam_mail_spam = --spam >>>> antispam_mail_notspam = --ham >>>> antispam_spam_pattern_ignorecase = spam;junk >>>> } >>>> --------------------- >>>> >>>> The permissions and execute bit are set in accordance with other >>>> executables in the same directory: >>>> >>>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>>> >>>> I would provide more log content, but the debug options seem to be gone >>>> in the forked, >= 2.0 version of this plug-in. >>>> >>>> Any ideas are very much appreciated. >>>> >>>> Thanks, >>>> >>>> -Ben >>> >>> Given that nobody shared any insight here, I decided to stand-up a >>> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >>> >>> While the error is different this time around, I am still unable to get >>> any variant of the dovecot-antispam plug-in working. >>> >>> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >>> I use the above configuration, my syslog fills-up with the following: >>> >>> dovecot: imap-login: Login: user=, method=PLAIN, >>> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >>> dovecot: master: Error: service(imap): child 2879 returned error 3 >>> >>> Surprisingly, I can find only two other mentions of "error 3" anywhere >>> on the Internet. Those mentions were from another user to this mailing list: >>> >>> http://www.dovecot.org/list/dovecot/2012-August/067787.html >>> http://www.dovecot.org/list/dovecot/2012-August/067814.html >>> >>> The lack of "error 3" ubiquity leads me to believe that I must be doing >>> something stupid. >>> >>> There's no question that this behavior is related to the >>> antispam-plugin, because when I commented-out the relevant lines in >>> Dovecot's configuration file, the error does not occur. >>> >>> I have followed the instructions at >>> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >>> result. >>> >>> The only instance in which I diverged from the Wiki article is that I >>> used the Ubuntu-provided dovecot-antispam package (instead of compiling >>> the plug-in from source). >>> >>> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >>> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >>> >>> I am not using the SpamAssassin daemon (spamd). >>> >>> I'm about to throw in the towel on this plug-in, so any help would be >>> greatly appreciated. >>> >>> Thank you, >>> >>> -Ben >> >> There are 2 versions of the dovecot-antispam package. The original >> version was created by Johannes Berg, then died almost a silent death, >> and was forked by Eugene Paskevitch. Later Johannes revived his original >> version somewhat. >> >> The ubuntu package is based on the 'revived Johannes fork' [1], while >> the documentation on the dovecot wiki describes the 'Eugene fork'. [2] >> >> [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright >> [2] http://wiki2.dovecot.org/Plugins/Antispam >> >> Thus you'll need to choose which plugin you want, then follow the >> complete procedure for that plugin only. They both can do what you want, >> but building and configuration are slightly different. >> >> It would be nice if Johannes and Eugene would either agree on a single >> version of the plugin, or rename one of the forks so they don't get >> mixed up that easily. I'm not sure if that has been discussed (on- or >> off-list) in the past. >> >> FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the >> 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to >> upgrade to 2.1 yet, so no experience there. >> >> -- >> Tom >> > > Thank you for your timely response, Tom. Much appreciated. > > Your remarks clear-up a lot of the confusion for me. I didn't realize > that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did > I realize that this is the version that is packaged with Ubuntu 12.04. > > Given that I'm more-or-less forced to work with Ubuntu LTS, I would > prefer to use the OS's package management system wherever possible. I > suppose, by extension, that I would prefer to use the "revived Johannes > fork". > > I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but > am unable to get either to work the way I want (mailtrain back-end with > real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki > article). Shouldn't the basic tenets of each version of the plug-in apply? > > Where should I start, in the way of seeking support? Basically, I intend > to use the Johannes version, but I want to configure it in the manner > described in the Eugene version Wiki article. > > I've already described (rather exhaustively) what happens when I attempt > to use the Johannes version and adapt the Eugene version instructions. > > I'm curious to know what method you've used over the years with such > glowing success. Which back-end did you use with Dovecot 1/Johannes? How > about with Dovecot 2/Eugene? > > Thanks again, > > -Ben > Thanks again, Tom. After you set me straight on the various versions of the plug-in, I was able to synthesize the relevant documentation and make it all work on Ubuntu 12.04 LTS with Dovecot 2. The key was to understand the following: 1.) The Ubuntu repositories use the Johannes versions of the plug-in (for both Dovecot 1 and 2). Ubuntu 10.04 LTS (Dovecot 1) http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html Ubuntu 12.04 LTS (Dovecot 2) http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to the Eugene version, and that version's configuration directives differ from the Johannes version 2 directives. So, anyone attempting to follow the Wiki article on Ubuntu will need to adapt the directives accordingly (see documentation links above for equivalent directive names). 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki instructions lacks a shebang; until a proper shebang is added to the shell script, script execution will fail with "status 1". I added the standard shebang: #!/bin/sh Here are the configuration directives that work for me: plugin { antispam_debug_target = syslog antispam_backend = pipe antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_pipe_tmpdir = /tmp antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham } My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. I'm using the following directives, but when I move mail from Inbox to Junk, for example, I see no evidence that antispam is taking action. plugin { antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_mail_tmpdir = /tmp antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham } Dovecot writes nothing at all to syslog. No errors, no reference to antispam. And the message is moved successfully (no failure/error notice in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", attempts to create a log file in /tmp, which works with Antispam 2. But no such luck here; nothing is ever written to /tmp. Version 2 of the plug-in includes a verbose debugging mode, but it seems that no such feature is present in version 1. I noticed also that version 1 seems not to require a back-end to be specified explicitly (as in version 2). Is the back-end implied via the configuration directives that are present? Respectfully, -Ben From f.bonnet at esiee.fr Thu Sep 20 18:57:17 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 20 Sep 2012 17:57:17 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B39BB.3060308@thelounge.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> Message-ID: <505B3CDD.6@esiee.fr> On 09/20/2012 05:43 PM, Reindl Harald wrote: > Am 20.09.2012 17:24, schrieb Surreal: >> Good Day, >> >> I'm using dovecot 2.1.8 and this is my dovecot -n output: >> >> # 2.1.8: dovecot.conf >> # OS: FreeBSD 9.0-STABLE i386 >> >> doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver >> >> connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix >> driver = mysql > no idea about BSD but on linux systems these are typically > seperated packages > > dovecot-2.1.10-4.fc17.20120919.rh.x86_64 > dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 > > > if you install from FreeBSD ports you need to enable the MYSQL backend during configuration ( make config ) From tss at iki.fi Thu Sep 20 19:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 20 Sep 2012 18:01:05 +0200 Subject: [Dovecot] doveadm with multiple commands Message-ID: This has been in my TODO for a while and today I implemented it (but didn't dare to test it yet :) So the problem has been that you want to run for example: doveadm expunge -A mailbox Trash savedbefore 7d doveadm purge -A This is annoying because it scans through the users' mailboxes twice, wasting disk IO and CPU. So how to make it do the scan only once? The syntax I thought of today would be: doveadm multi [-A | -u wildcards] [ [...]] So the first two commands could be for example: doveadm multi -A : expunge mailbox Trash savedbefore 7d : purge The ':' character is the separator. Any other string could be used as well, for example "?SEP?" or whatever. Thoughts? Any better name for the command than "multi"? From horsley1953 at gmail.com Thu Sep 20 15:11:26 2012 From: horsley1953 at gmail.com (Tom Horsley) Date: Thu, 20 Sep 2012 08:11:26 -0400 Subject: [Dovecot] Sieve as mail client? Message-ID: <20120920081126.0627cd71@tomh> A slightly off topic question, but maybe someone here has seen something like this: I want a program that operates entirely on the client side of an IMAP server, and implements the sieve filtering language by doing the necessary client side operations to move mail on the IMAP server, expunge mail, etc. It would also act as a proxy between my real mail client and the IMAP server so I'd never see any mail that hasn't been filtered first. The benefits are obvious: I could finally get sieve filtering on my stupid corporate exchange mail server :-). From pixilla at macports.org Thu Sep 20 19:12:25 2012 From: pixilla at macports.org (Bradley Giesbrecht) Date: Thu, 20 Sep 2012 09:12:25 -0700 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. Is there a Johannes version 2? I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. http://johannes.sipsolutions.net/files/antispam.html Regards, Bradley Giesbrecht (pixilla) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2763 bytes Desc: not available URL: From jerry at seibercom.net Thu Sep 20 19:12:42 2012 From: jerry at seibercom.net (Jerry) Date: Thu, 20 Sep 2012 12:12:42 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3CDD.6@esiee.fr> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> Message-ID: <20120920121242.120102d6@scorpio> On Thu, 20 Sep 2012 17:57:17 +0200 Frank Bonnet articulated: > if you install from FreeBSD ports you need to enable > the MYSQL backend during configuration ( make config ) If you don't install Dovecot via the FreeBSD ports system, you are just asking for problems. As previously stated, do run "make config" and perhaps "make config-recursive" prior to actually attempting to build the port. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ben at indietorrent.org Thu Sep 20 19:33:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 12:33:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> Message-ID: <505B454D.6000402@indietorrent.org> On 9/20/2012 12:12 PM, Bradley Giesbrecht wrote: > > On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > >> Thanks again, Tom. After you set me straight on the various versions of >> the plug-in, I was able to synthesize the relevant documentation and >> make it all work on Ubuntu 12.04 LTS with Dovecot 2. >> >> The key was to understand the following: >> >> 1.) The Ubuntu repositories use the Johannes versions of the plug-in >> (for both Dovecot 1 and 2). >> >> Ubuntu 10.04 LTS (Dovecot 1) >> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html >> >> Ubuntu 12.04 LTS (Dovecot 2) >> http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html >> >> 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to >> the Eugene version, and that version's configuration directives differ >> from the Johannes version 2 directives. > > Is there a Johannes version 2? Thanks for your reply, Bradley. That seems to be the implication here: http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html Johannes is listed as an author, the package name is dovecot-antispam_2.0+20120225, and the manpage is nearly identical to that of version 1 at http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html . > I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > >> So, anyone attempting to follow >> the Wiki article on Ubuntu will need to adapt the directives accordingly >> (see documentation links above for equivalent directive names). >> >> 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki >> instructions lacks a shebang; until a proper shebang is added to the >> shell script, script execution will fail with "status 1". I added the >> standard shebang: >> >> #!/bin/sh >> >> Here are the configuration directives that work for me: >> >> plugin { >> antispam_debug_target = syslog >> antispam_backend = pipe >> antispam_signature = X-DSPAM-Signature >> antispam_signature_missing = error >> antispam_spam_pattern_ignorecase = SPAM;JUNK >> antispam_pipe_tmpdir = /tmp >> antispam_pipe_program = /usr/bin/sa-learn-pipe.sh >> antispam_pipe_program_spam_arg = --spam >> antispam_pipe_program_notspam_arg = --ham >> } >> >> My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. > http://johannes.sipsolutions.net/files/antispam.html This documentation seems to apply to the Dovecot 2 version; the antispam_backend directive doesn't seem to exist in version 1, at least according to the official Ubuntu package documentation (cited above). In fact, this appears to be the exact document that is used for the Ubuntu package for Dovecot 2 (also cited above). > > Regards, > Bradley Giesbrecht (pixilla) > From surreal at bsdmail.com Thu Sep 20 19:35:08 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 00:35:08 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <20120920121242.120102d6@scorpio> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> Message-ID: <505B45BC.8020109@bsdmail.com> On 9/21/2012 12:12 AM, Jerry wrote: > On Thu, 20 Sep 2012 17:57:17 +0200 > Frank Bonnet articulated: > >> if you install from FreeBSD ports you need to enable >> the MYSQL backend during configuration ( make config ) > If you don't install Dovecot via the FreeBSD ports system, you are just > asking for problems. As previously stated, do run "make config" and > perhaps "make config-recursive" prior to actually attempting to build > the port. > I did, MySQL is ticked when i build dovecot2 from /usr/ports/mail/dovecot2/ From ml at cibest.com Thu Sep 20 20:04:27 2012 From: ml at cibest.com (Cibest ML) Date: Thu, 20 Sep 2012 19:04:27 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> Message-ID: <505B4C9B.5080708@cibest.com> Hi Timo, I've finally solved the issue. As usual, thinking out of the box helped :) In fact, it was conf related but not where it was expected. There was an autocreate = Inbox & autosubscribe = Inbox in 90-plugin.conf Having this configuration seems to double count the Inbox quota during quota recalc (but not during normal IMAP operations like moving or receiving a message). Might be some kind of bug somewhere in recalc. The strange thing is that another more simple dovecot system we have (no nfs, all imap/smtp/filtering operations done on the server) also has this plugin conf but doesn't count quota twice. Anyway, after running a small script to recalc all the quota, everything is now fine. Thanks for you help -- Regards S?bastien On 19/09/2012 14:47, Timo Sirainen wrote: > On 19.9.2012, at 15.15, Cibest ML wrote: > >> There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). > No idea. INBOX is treated case-insensitively by IMAP protocol. > >> I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) > Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. > From daryl at isletech.net Thu Sep 20 21:42:05 2012 From: daryl at isletech.net (Daryl Richards) Date: Thu, 20 Sep 2012 14:42:05 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B45BC.8020109@bsdmail.com> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> Message-ID: <505B637D.4080901@isletech.net> On 12-09-20 12:35 PM, Surreal wrote: > On 9/21/2012 12:12 AM, Jerry wrote: >> On Thu, 20 Sep 2012 17:57:17 +0200 >> Frank Bonnet articulated: >> >>> if you install from FreeBSD ports you need to enable >>> the MYSQL backend during configuration ( make config ) >> If you don't install Dovecot via the FreeBSD ports system, you are just >> asking for problems. As previously stated, do run "make config" and >> perhaps "make config-recursive" prior to actually attempting to build >> the port. >> > I did, MySQL is ticked when i build dovecot2 from > /usr/ports/mail/dovecot2/ If you do dovecot --build-options, does it list mysql? root at mail:~# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql -- Daryl Richards Isle Technical Services Inc. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 20 23:17:41 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 20 Sep 2012 22:17:41 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <20120920201741.GA9646@daniel.localdomain> Timo Sirainen wrote: > doveadm multi [-A | -u wildcards] [ [...]] > > Thoughts? Would be nice if this also worked through doveadm director proxy and executes the command sequence on the correct backend host for each user. As command name I could also think of "doveadm sequence", which implies the commands being executed in serial order. Regards Daniel -- https://plus.google.com/103021802792276734820 From ben at indietorrent.org Fri Sep 21 00:09:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 17:09:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <505B85FD.4000604@indietorrent.org> On 9/20/2012 11:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > I'm using the following directives, but when I move mail from Inbox to > Junk, for example, I see no evidence that antispam is taking action. > > plugin { > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_mail_tmpdir = /tmp > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > } > > Dovecot writes nothing at all to syslog. No errors, no reference to > antispam. And the message is moved successfully (no failure/error notice > in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", > attempts to create a log file in /tmp, which works with Antispam 2. But > no such luck here; nothing is ever written to /tmp. > > Version 2 of the plug-in includes a verbose debugging mode, but it seems > that no such feature is present in version 1. > > I noticed also that version 1 seems not to require a back-end to be > specified explicitly (as in version 2). Is the back-end implied via the > configuration directives that are present? > > Respectfully, > > -Ben Just a follow-up to report that I finally have the Antispam plug-in (the Johannes version) -- with the mailtrain/pipe back-end -- working as expected in both of the following environments: - Ubuntu 10.04 with Dovecot 1.2.9 - Ubuntu 12.04 with Dovecot 2.0.19 In addition to the points that I mentioned in my previous note, a crucial step in the debugging process was to modify the "sa-learn-pipe.sh" script to produce more verbose output. Unlike Antispam 2, Antispam 1 lacks a verbose debugging mode, so when the plug-in does not work as expected, tracking-down the root cause can be much more difficult. Here's the shell script from http://wiki2.dovecot.org/Plugins/Antispam , with my modifications: http://pastebin.com/igTXCYQF After making these modifications to the shell script, I was able to determine the cause of the failure in Dovecot 1.2.9. In fact, I was incorrect earlier when I stated that version 2 was working; the same problem applied there, too. The "vmail" user, as whom the script is executed, lacks the permissions required to change the Bayes token database (this is a "good thing"). An excerpt from the custom log file: 29722-start (--spam) Calling (as user vmail) '/usr/bin/sa-learn --spam /tmp/sendmail-msg-29722.txt' bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 493. Learned tokens from 0 message(s) (1 message(s) examined) 29722-end The only means by which I was able to work around this issue -- without changing filesystem permissions, users, groups, or otherwise compromising system security -- was to migrate from the standard flat-file Bayes database format to MySQL. (The guide at http://www.starbridge.org/spip/spip.php?article15 had everything I needed, and it only took me a few minutes to implement.) After making that change, everything began working as expected; here's a sample custom log entry: 30565-start (--ham) Calling (as user vmail) '/usr/bin/sa-learn --ham /tmp/sendmail-msg-30565.txt' Learned tokens from 1 message(s) (1 message(s) examined) 30565-end In any event, this concludes the saga, and I must say, I'm thrilled to have it all working. Thanks again to everyone who contributed valuable input and pointed me in the right direction. And thanks to Johannes and Eugene (and everyone else who contributed to Antispam) for solving a major hurdle in Bayes DB training. -Ben From surreal at bsdmail.com Fri Sep 21 04:35:22 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 09:35:22 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B637D.4080901@isletech.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> <505B637D.4080901@isletech.net> Message-ID: <505BC45A.1010003@bsdmail.com> On 9/21/2012 2:42 AM, Daryl Richards wrote: > On 12-09-20 12:35 PM, Surreal wrote: >> On 9/21/2012 12:12 AM, Jerry wrote: >>> On Thu, 20 Sep 2012 17:57:17 +0200 >>> Frank Bonnet articulated: >>> >>>> if you install from FreeBSD ports you need to enable >>>> the MYSQL backend during configuration ( make config ) >>> If you don't install Dovecot via the FreeBSD ports system, you are just >>> asking for problems. As previously stated, do run "make config" and >>> perhaps "make config-recursive" prior to actually attempting to build >>> the port. >>> >> I did, MySQL is ticked when i build dovecot2 from >> /usr/ports/mail/dovecot2/ > If you do dovecot --build-options, does it list mysql? > > root at mail:~# dovecot --build-options > Build options: ioloop=kqueue notify=kqueue ipv6 openssl > io_block_size=8192 > Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw > SQL drivers: mysql > Passdb: checkpassword pam passwd passwd-file sql > Userdb: checkpassword nss passwd prefetch passwd-file sql > At first i have noticed that i dont have mysql when i did --build-options, so i tried to deinstall and reinstall dovecot2 port and now i have this also: hostname# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql But i still keep getting the same error hostname# service dovecot start doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver Starting dovecot. doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot Still the same output for doveconf -n hostname# doveconf -n # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = ren.wf auth_mechanisms = plain login auth_realms = ren.wf graphics.re disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } doveconf: Error: userdb is missing driver protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver From tss at iki.fi Fri Sep 21 08:28:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 21 Sep 2012 07:28:57 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <20120920201741.GA9646@daniel.localdomain> References: <20120920201741.GA9646@daniel.localdomain> Message-ID: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> On 20.9.2012, at 22.17, Daniel Parthey wrote: > Timo Sirainen wrote: >> doveadm multi [-A | -u wildcards] [ [...]] >> >> Thoughts? > > Would be nice if this also worked through doveadm director proxy > and executes the command sequence on the correct backend host for > each user. Yes, it's going to. > As command name I could also think of "doveadm sequence", which > implies the commands being executed in serial order. Hmm. Maybe. From alec at alec.pl Fri Sep 21 11:23:02 2012 From: alec at alec.pl (A.L.E.C) Date: Fri, 21 Sep 2012 10:23:02 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <505C23E6.2070606@alec.pl> On 09/20/2012 06:01 PM, Timo Sirainen wrote: > Thoughts? Any better name for the command than "multi"? How about 'execute' or 'exec'. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From tomas at blueboard.cz Fri Sep 21 12:32:37 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Fri, 21 Sep 2012 11:32:37 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? Message-ID: <505C3435.30601@blueboard.cz> Hello, I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or dovecot<>freebsd bug? I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over 1000 processes with kernel message: maxproc limit exceeded by uid 89 Could anybody help? Many thanks Tomas My system have following settings: FreeBSD 9.0 / AMD64 Dovecot 2.1.8 kern.maxproc: 12288 kern.maxfilesperproc: 36864 kern.maxprocperuid: 8192 no limit for uid 89: # limit cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited My dovecot.conf: # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 2048 default_process_limit = 2048 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /data/logfiles/dovecot/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_plugins = " quota" mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore sieve = ~/.sieve/dovecot.sieve sieve_dir = ~/.sieve } protocols = imap pop3 sieve service anvil { client_limit = 6147 } service auth { client_limit = 8192 unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 2048 service_count = 1 } service imap { client_limit = 1 process_limit = 2048 } service managesieve { process_limit = 2048 } service pop3-login { process_limit = 2048 service_count = 1 } service pop3 { client_limit = 1 process_limit = 2048 } shutdown_clients = no ssl_cert = object mailstore In-Reply-To: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> Message-ID: On 17 September 2012 13:57, Timo Sirainen wrote: > On 14.9.2012, at 18.16, Damien Churchill wrote: > >> On 14 September 2012 15:59, Timo Sirainen wrote: >>> On 14.9.2012, at 17.46, Damien Churchill wrote: >>> >>>> I was wondering what would be entailed in modifying dovecot 2.2 to >>>> support storing mail in an object store. I've seen a few mails dotted >>>> around in the ML history about supporting such a thing and seen it's >>>> basically dependant upon some changes in lib-storage to support >>>> writing messages without locking. Is this still the case? >>> >>> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >>> >> >> That's good to hear :) I've been evaluating RADOS as an object store, >> which is similar to S3. Although any distributed storage would be >> great. I'd be more than happy to help code it! > > I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. > >>> The generic idea is: >>> - only one server accesses one user simultaneously >>> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >>> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >>> - support high latency: asynchronous reads/writes. prefetch mail bodies. >> >> I'm assuming that the director would be used in order to distribute >> connections to the same server, so it's only within a local instance >> of dovecot you'd need to be aware of what currently has a connection >> open for that user? > > Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > Sounds good! I'll spend some time digging through the source code getting familiar. >> How are you planning on handling the situation where say node X dies >> and hasn't uploaded the latest index file? Would that result in >> missing messages from the mailbox when accessed by another node, or is >> the local index intended to be more of a write-through cache? > > No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) Part of me thinks making this configurable might be a good idea depending upon what the installation is trying to achieve. Since the recovery process will need to be implemented regardless allowing the user to configure dovecot to perform a write to both the local and object index when a flag or something else is modified. Another thought occurs to me, when using the LDA how will that be able to update the index upon delivery of a new message if another node is currently accessing the mailbox? From h.reindl at thelounge.net Fri Sep 21 12:52:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 11:52:50 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505C38F2.40402@thelounge.net> Am 21.09.2012 11:32, schrieb Tom?? Randa: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried > everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or > dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over > 1000 processes with kernel message: no idea about BSD but your config are a total of up to 10.240 PROCESSES one process has MUCH MORE than one file-handle i have ONE imap-login process with 572 file-handles your configuration eats up to 5 Mio. file-handles maybe you running out of OS ressources 1000 prcoesses are up to 500000 file handles for one service [root at mail:~]$ ps aux | grep imap-login | wc -l 2 [root at mail:~]$ lsof | grep imap-l | wc -l 572 ___________________ for imaplogin / pop3-login as example you do not need a PROCESS per connection service_count = 0 process_min_avail = 1 process_limit = 10 client_limit = 200 this can handle 2000 connections with up to 10 processes > service imap-login { > process_limit = 2048 > service_count = 1 > } > service imap { > client_limit = 1 > process_limit = 2048 > } > service managesieve { > process_limit = 2048 > } > service pop3-login { > process_limit = 2048 > service_count = 1 > } > service pop3 { > client_limit = 1 > process_limit = 2048 > } -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From trashcan at odo.in-berlin.de Fri Sep 21 14:48:45 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 21 Sep 2012 13:48:45 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: On 2012-09-21 10:23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. doveadm execall [...] Regards, Michael From matthias at couch-blog.de Fri Sep 21 16:09:06 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:09:06 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? Message-ID: <505C66F2.7060704@couch-blog.de> Hello! Could it be, that http://xi.rename-it.nl is down? Thanks, Matthias From h.reindl at thelounge.net Fri Sep 21 16:22:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:22:03 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C66F2.7060704@couch-blog.de> References: <505C66F2.7060704@couch-blog.de> Message-ID: <505C69FB.3040402@thelounge.net> Am 21.09.2012 15:09, schrieb Matthias: > Hello! > > Could it be, that http://xi.rename-it.nl is down? not from vienna but what the hell has this question to search on the dovecot-list? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Fri Sep 21 16:27:37 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 21 Sep 2012 15:27:37 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132737.GX9342@charite.de> * Reindl Harald : > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? Isn't that where the sieve extension is located at? http://xi.rename-it.nl/debian/dists/unstable-auto/ -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mailinglist at darac.org.uk Fri Sep 21 16:27:47 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 21 Sep 2012 14:27:47 +0100 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132747.GA23967@darac.org.uk> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? > The mentioned site is a host for Prebuilt Binaries: http://wiki2.dovecot.org/PrebuiltBinaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 21 16:40:51 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:40:51 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <20120921132747.GA23967@darac.org.uk> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> Message-ID: <505C6E63.6020608@thelounge.net> Am 21.09.2012 15:27, schrieb Darac Marjal: > On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >> >> >> Am 21.09.2012 15:09, schrieb Matthias: >>> Hello! >>> >>> Could it be, that http://xi.rename-it.nl is down? >> >> not from vienna >> >> but what the hell has this question to search >> on the dovecot-list? >> > > The mentioned site is a host for Prebuilt Binaries: > http://wiki2.dovecot.org/PrebuiltBinaries ok, this should have been mentioned because only few people understand "Welkom op de website van Rename-it, uw partner in systeembeheer en internetoplossingen." * most people using packages from their distribution * the rest is compiling from source * one or two others are using foreign binaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From sb at dod.no Fri Sep 21 16:53:29 2012 From: sb at dod.no (Steinar Bang) Date: Fri, 21 Sep 2012 15:53:29 +0200 Subject: [Dovecot] doveadm with multiple commands References: Message-ID: <87mx0jbhye.fsf@dod.no> >>>>> Timo Sirainen : > Thoughts? How about: a) Dropping into a command prompt where it's possible to give commands? b) Making it possible to feed commands through a pipe (ie reading lines from stdin)? From matthias at couch-blog.de Fri Sep 21 16:55:48 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:55:48 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C6E63.6020608@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> <505C6E63.6020608@thelounge.net> Message-ID: <505C71E4.6070105@couch-blog.de> Am 21.09.2012 15:40, schrieb Reindl Harald: > > Am 21.09.2012 15:27, schrieb Darac Marjal: >> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >>> >>> Am 21.09.2012 15:09, schrieb Matthias: >>>> Hello! >>>> >>>> Could it be, that http://xi.rename-it.nl is down? >>> not from vienna >>> >>> but what the hell has this question to search >>> on the dovecot-list? >>> >> The mentioned site is a host for Prebuilt Binaries: >> http://wiki2.dovecot.org/PrebuiltBinaries > ok, this should have been mentioned because only few > people understand "Welkom op de website van Rename-it, > uw partner in systeembeheer en internetoplossingen." > > * most people using packages from their distribution > * the rest is compiling from source > * one or two others are using foreign binaries > Oh now all is great, thanks for small help ;) I test a little bit with a small sieve skript, to filter my mails on my mailserver with dovecot 2.1. Now it just works. Sorry if the question was on the wrong mailinglist. From alessio at skye.it Fri Sep 21 17:13:15 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 21 Sep 2012 16:13:15 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <1347680376.5717.61.camel@maclinux> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <1347680376.5717.61.camel@maclinux> Message-ID: <505C75FB.4080006@skye.it> Il 15/09/2012 05:39, Jeff Gustafson ha scritto: > On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: >> I've a whole new design for it and I was planning on implementing it >> for v2.2. Do you want to help coding it? :) Which storage would you >> want to use? >> >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and >> accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split >> brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > With this system, would the read/write ultimately go to a normal OS > file function? If it is a file function, could this be used with a > system like glusterfs, ceph, etc? The other option would be to write it > against a object store client library and bypass the normal file > functions. > > ...Jeff > Also other users are talking about Ceph and Dovecot http://www.mail-archive.com/ceph-devel at vger.kernel.org/msg07345.html -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stan at hardwarefreak.com Sat Sep 22 00:16:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 21 Sep 2012 16:16:50 -0500 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505CD942.7000104@hardwarefreak.com> On 9/21/2012 4:32 AM, Tom?? Randa wrote: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. What is your value for kern.maxusers? Did you try increasing it? Note in the 2nd paragraph below the relationship between kern.maxusers and process limit. From what you describe it would seem you have a process limit of 1044, thus a kern.maxusers value of 64. Considering your manual setting of 8192 processes is apparently being ignored, it would seem the kern.maxusers value is causing it to be overridden. From: http://www.pl.freebsd.org/doc/handbook/configtuning-kernel-limits.html As of FreeBSD 4.5, kern.maxusers is automatically sized at boot based on the amount of memory available in the system, and may be determined at run-time by inspecting the value of the read-only kern.maxusers sysctl. Some sites will require larger or smaller values of kern.maxusers and may set it as a loader tunable; values of 64, 128, and 256 are not uncommon. We do not recommend going above 256 unless you need a huge number of file descriptors; many of the tunable values set to their defaults by kern.maxusers may be individually overridden at boot-time or run-time in /boot/loader.conf (see the loader.conf(5) man page or the /boot/defaults/loader.conf file for some hints) or as described elsewhere in this document. Systems older than FreeBSD 4.4 must set this value via the kernel config(8) option maxusers instead. In older releases, the system will auto-tune maxusers for you if you explicitly set it to 0[1]. When setting this option, you will want to set maxusers to at least 4, especially if you are using the X Window System or compiling software. The reason is that the most important table set by maxusers is the maximum number of processes, which is set to 20 + 16 * maxusers, so if you set maxusers to 1, then you can only have 36 simultaneous processes, including the 18 or so that the system starts up at boot time and the 15 or so you will probably create when you start the X Window System. Even a simple task like reading a manual page will start up nine processes to filter, decompress, and view it. Setting maxusers to 64 will allow you to have up to 1044 simultaneous processes, which should be enough for nearly all uses. If, however, you see the dreaded proc table full error when trying to start another program, or are running a server with a large number of simultaneous users (like ftp.FreeBSD.org), you can always increase the number and rebuild. -- Stan From durket at highwire.stanford.edu Sat Sep 22 00:54:02 2012 From: durket at highwire.stanford.edu (Michael Durket) Date: Fri, 21 Sep 2012 14:54:02 -0700 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? Message-ID: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? From robert at schetterer.org Sat Sep 22 08:46:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 07:46:15 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> Message-ID: <505D50A7.3090003@schetterer.org> Am 21.09.2012 23:54, schrieb Michael Durket: > I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): > > Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) > > Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? > you should upgrade to recent stable version 2.1.x -- Best Regards MfG Robert Schetterer From tlhackque at yahoo.com Sat Sep 22 15:04:22 2012 From: tlhackque at yahoo.com (tlhackque) Date: Sat, 22 Sep 2012 08:04:22 -0400 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" Message-ID: <505DA946.90409@yahoo.com> Dovecot 2.1.10 Client = Thunderbird. Local disks. mbox format. Attempted to rename a folder, failed with: CANNOT Renaming not supported across conflicting directory permissions I don't understand why this should happen. o I was renaming within a directory - the attempt was to rename "Domain Names &- SSL Certificates" to "Domain Names". No directory was specified o The reason the directory has g=s is for dovecot - so that the 'mail' group (12) will stick to any files created. I was getting more understandable permission failures before I added it. Since 's' doesn't apply to files, I don't see why this should be viewed as a conflict. o If I chmod g-s the directory, the command succeeds. But then I'm back to file ownership problems... drwxrws--- 4 4503 12 4096 Sep 22 06:17 F/N/ -rw-rw---- 1 4503 12 81530 Sep 22 06:17 F/N/Domain Names &- SSL Certificates Sequence from TB's trace (crlfs for clarity): 1140[11ebcf00]: 121e8c00:imap.example.com:A:SendData: 66 rename "F/N/Domain Names &- SSL Certificates" "F/N/Domain Names" 1140[11ebcf00]: ReadNextLine [stream=12135a28 nb=80 needmore=0] 1140[11ebcf00]: 121e8c00:imap.example.com:A:CreateNewLineFromSocket: 66 NO [CANNOT] Renaming not supported across conflicting directory permissions 9592[c7d6840]: ReadNextLine [stream=ab687e8 nb=22 needmore=0] Config: # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.22.14-72.fc6 i686 Fedora Core release 6 (Zod) first_valid_gid = 4000 first_valid_uid = 4000 hostname = smtp.example.com login_greeting = Dovecot ready. Unauthorized access is prohibited. mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { driver = pam } service imap-login { inet_listener imap { address = imap.v4.example.com imap.v6.example.com port = 143 } inet_listener imaps { address = imap.v4.example.com imap.v6.example.com port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { address = pop.v4.example.com pop.v6.example.com port = 110 } inet_listener pop3s { address = pop.v4.example.com pop.v6.example.com port = 995 ssl = yes } } ssl_ca = Hi, running in problems with KMail I decided to switch to dovecot-imap & thunderbird on my kubuntu 12.04 64 bit (so I can read my imap-folder-mails with KMail again if it gets more stable) What I did: - I wrote a python script to convert my KMail mails folder to a thunderbird mailfolder (directory tree containing mbox files and *.sbd folders) - I created a fake pop mail account in thunderbird and imported the whole mail-structure (i just copied it) - I create a local imap account ( so I have a ~/Maildir dovecot folder in my user dir) - In /etc/dovecot: 1. /etc/dovecot/10-mail.conf (line 10) I set: mail_location = maildir:~/Maildir:LAYOUT=fs 2. /etc/dovecot/20-imap.conf (line 57) I set: imap_client_workarounds = tb-extra-mailbox-sep Now I want to copy the whole mailstructure from my imported fake pop mailfolder to the dovecot imap folder. What happens: Lets say my source structure is mailfolder1 + mail1 + mail2 + subMailFolder2 + mail1-in-subMailFolder2 + mail2-in-subMailFolder2 + subSubMailFolder3 + mail1-in-subMailFolder3 + mail2-in-subMailFolder3 I thunderbird (version 15) I drag and drop my mailFolder1 to the imap folder and I get: mailfolder1 + mail1 + mail2 + subMailFolder2 + subSubMailFolder3 as you See only the mails of the *TOP* folder are copied. All subfolders are created but they contain NO MAILS. Did somebody experience the same problem??? Otherwise I will write a python script to do that (I dont hope to need to write a mailclient in python ;-) ) Any hint is greatly welcome. Anton From me at junc.org Sat Sep 22 16:57:54 2012 From: me at junc.org (Benny Pedersen) Date: Sat, 22 Sep 2012 15:57:54 +0200 Subject: [Dovecot] =?utf-8?q?Dovecot_1=2E2=2E11_panic_in_mail-transaction-?= =?utf-8?q?log-view=3F?= In-Reply-To: <505D50A7.3090003@schetterer.org> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: Robert Schetterer skrev den 2012-09-22 07:46: > you should upgrade to recent stable version 2.1.x and this is a simple change from 1.x to 2.x :( i have never a panic in 1.x here From robert at schetterer.org Sat Sep 22 17:03:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 16:03:12 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: <505DC520.9050505@schetterer.org> Am 22.09.2012 15:57, schrieb Benny Pedersen: > Robert Schetterer skrev den 2012-09-22 07:46: > >> you should upgrade to recent stable version 2.1.x > > and this is a simple change from 1.x to 2.x :( depends on how complex your setup is youre should really try upgrade http://wiki.dovecot.org/Upgrading/1.2 http://wiki2.dovecot.org/Upgrading/2.0 > > i have never a panic in 1.x here > yeah, but i guess active bug supported are only recent versions, however wait till Timos answers > > > -- Best Regards MfG Robert Schetterer From csch at kiez.net Sat Sep 22 19:11:06 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 18:11:06 +0200 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 Message-ID: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Ok, here's a toughie: Out of a whim (and because of the bad weather) I today decided to upgrade my completely functioning 2.0.15 installation on my Mac OS X 10.6.7 system. NB: It's not a Mac OS X "Server", as sold by Apple - I have compiled my dovecots myself for quite a while. Anyhow: At first everything appeared to work fine after the upgrade. Up until I created a new user and -while testing- SELECTed his INBOX. The SELECT was "stuck". Though the process seemed to be alive I could only get rid of it with a "kill -9 ". The same happened when selecting a newly created mailbox on any of the other (otherwise functioning) accounts, so the problem had nothing to do with the new account, but rather with the fact, that it's INBOX was empty. Using dtruss and gdb I found out, that the dovecot process was trying to obtain a GUID and in the course of doing so invoked gethostbyname(), which intern caused a lot of "mach message" handling and somewhere, deep down there, the process was stuck. Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: sudo /usr/libexec/StartupItemContext dovecot (again: all this was working fine under 2.0.15) Now with 2.1.10, when I manually invoke dovecot with just sudo dovecot Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. Now -with all that said- here's my question: What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? It appears, that the imap process no longer "inherits" the StartupItemContext from the main process, so some change between 2.0.15 and 2.1.10 must have broken it ... Any help is highly appreciated - Clemens PS: I google'd around a lot and searched the mailing-lists, of course. I only found a post of someone who ran into the same/similar problem dating back to Feb 21st 2012 under the subject "dovecot freezes when trying to get mail from maildir with mail", but it was quickly dismissed without ever getting resolved and that was that. PS2: I intentionally didn't include any configs with this mail as they seem to be irrelevant, but of course I can generate the necessary output if needed. From tss at iki.fi Sat Sep 22 19:50:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:50:27 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> On 21.9.2012, at 11.23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. From tss at iki.fi Sat Sep 22 19:52:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:52:39 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <87mx0jbhye.fsf@dod.no> References: <87mx0jbhye.fsf@dod.no> Message-ID: On 21.9.2012, at 16.53, Steinar Bang wrote: >>>>>> Timo Sirainen : > >> Thoughts? > > How about: > a) Dropping into a command prompt where it's possible to give commands? > b) Making it possible to feed commands through a pipe (ie reading lines > from stdin)? Both of these would require adding some kind of new command parameter parsing. Currently shell does all the "multi word string" parsing and escaping and such. My original thought was also to be able to read commands from stdin, but I'd rather not add such special command parser, at least not yet.. From jcblanco at fi.upm.es Sat Sep 22 20:29:46 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Sat, 22 Sep 2012 19:29:46 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <505A09D0.6040607@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> <505A09D0.6040607@rename-it.nl> Message-ID: <505DF58A.3020403@fi.upm.es> Sorry I've sent a config.log but get retained for approval. This is the beginning of the file: This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by Pigeonhole configure 0.3.3, which was generated by GNU Autoconf 2.59. Invocation command line was $ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --target=x86_64-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-dovecot=/home/jc/rpmbuild/BUILD/dovecot-2.1.10 --with-managesieve=yes --enable-header-install=yes INSTALL_DATA=install -c -p -m644 ## --------- ## ## Platform. ## ## --------- ## hostname = testv.fi.upm.es uname -m = x86_64 uname -r = 2.6.18-308.13.1.el5debug uname -s = Linux uname -v = #1 SMP Tue Aug 21 18:34:48 EDT 2012 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = x86_64 /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown Regards Juan C. Blanco On 19/09/2012 20:07, Stephan Bosch wrote: > On 9/19/2012 6:22 PM, Juan C. Blanco wrote: >> On 18/09/2012 20:49, Stephan Bosch wrote: >>> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>>> It seems that the error is related to the ld.gold fix, If I revert the >>>> c52a0c561311 patch the error is not produced >>> >>> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >>> but I forgot the Dovecot v2.1 tree. Fixed now: >>> >>> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > D'oh, this is the v2.2 change. The URL I meant is this one: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >>> >>> This only happens when you compile against installed Dovecot headers, >>> people using --with-dovecot= (like me) would not have seen this. >>> >>> I'll push out a new release once 2.1.10 comes out. I'm being told that >>> is imminent. >> >> Sorry, but using the new 0.3.3 version I'm getting the same error and >> I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in >> configure, not the installed headers. If needed I can send you the >> configure invocation command. > > Yes, that could be helpful. > > Regards, > > Stephan. > From csch at kiez.net Sat Sep 22 22:51:22 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 21:51:22 +0200 Subject: [Dovecot] Update: Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 (and 2.0.21) References: <692EAFE7-E424-4F95-8DFA-C963B4957E11@kiez.net> Message-ID: Update: I tried with 2.0.21 and this also works just fine. So it must be something which came in with 2.1.x PS: Where is the documentaion for 2.1.x - i.e. for all the nice additions Timo made? The Website "only" has 2.0.x, as far as I can tell? (might be wrong here - hadn't touched the whole thing for a while, as my "civil life" had occupied me :-) Greetings, Clemens From joe at netmusician.org Sat Sep 22 22:52:42 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:52:42 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E170A.5090800@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sat Sep 22 22:57:03 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:57:03 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E180F.5060407@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. From mr88talent at gmail.com Sun Sep 23 04:00:21 2012 From: mr88talent at gmail.com (Gary V) Date: Sat, 22 Sep 2012 19:00:21 -0600 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On Sat, Sep 22, 2012 at 1:57 PM, Joe Auty wrote: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I can > try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp >> 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple SSL > certs. May not be related at all, but when I see segfaults on Debian, the first thing I try is to switch to bash as the default system shell: dpkg-reconfigure dash Answer: Use dash as the default system shell (/bin/sh)? -- Gary V From joe at netmusician.org Sun Sep 23 07:58:54 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 00:58:54 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505E970E.1040803@netmusician.org> > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sun Sep 23 08:03:28 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 01:03:28 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E970E.1040803@netmusician.org> References: <505E180F.5060407@netmusician.org> <505E970E.1040803@netmusician.org> Message-ID: <505E9820.5070007@netmusician.org> > > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. P.S. sorry for the duplicate message, it seems to have something to do with attaching images to messages. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ > From robert at schetterer.org Sun Sep 23 08:44:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 07:44:36 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: <505EA1C4.2000808@schetterer.org> Am 22.09.2012 21:57, schrieb Joe Auty: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I > can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 >> sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple > SSL certs. show your dovecot conf -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 09:00:10 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 02:00:10 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA1C4.2000808@schetterer.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> Message-ID: <505EA56A.6010102@netmusician.org> > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf Thanks! http://pastebin.com/gVRrCjyK I took what you said literally, please let me know if you'd like to see any of the other Debian config files. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org From weber at zackbummfertig.de Sun Sep 23 10:56:52 2012 From: weber at zackbummfertig.de (weber at zackbummfertig.de) Date: Sun, 23 Sep 2012 09:56:52 +0200 Subject: [Dovecot] Dovecot Clustering with dsync over ssh Message-ID: Hello List, i only found an old post of tiemo where he announced that he will do an wiki article how to setup dovecot cluster with dsync over ssh. anyone knows if this artcile exists and where? i would like to setup a dovecot cluster with 2 machines. thanks for help marko From dovecot-list at mohtex.net Sun Sep 23 12:07:29 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Sun, 23 Sep 2012 16:07:29 +0700 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505ED151.8090807@mohtex.net> Joe Auty wrote the following on 23.09.2012 13:00: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > With "show your dovecot conf" Robert means show us the output of "doveconf -n" From joe at netmusician.org Sun Sep 23 12:16:15 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 05:16:15 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED151.8090807@mohtex.net> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> Message-ID: <505ED35F.1020402@netmusician.org> > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: From tss at iki.fi Sun Sep 23 12:58:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 23 Sep 2012 12:58:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On 22.9.2012, at 22.57, Joe Auty wrote: > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work: service imap-login { executable = imap-login -D } Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full From robert at schetterer.org Sun Sep 23 13:27:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 12:27:15 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505EE403.8060209@schetterer.org> Am 23.09.2012 08:00, schrieb Joe Auty: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK fine, but what i/we like to see is the result of dovecot -n on the list, and pleady avoid mail your jpg only to the list ( what ever this is ) > > I took what you said literally, please let me know if you'd like to see > any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I >> suspect this might be causing performance issues. Any suggestions what >> I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 13:44:23 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 06:44:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED35F.1020402@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> <505ED35F.1020402@netmusician.org> Message-ID: <505EE807.40903@netmusician.org> > > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From joe at netmusician.org Sun Sep 23 14:05:36 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 07:05:36 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505EED00.6090109@netmusician.org> > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full I hope I'm doing this correctly! # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...(no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f789cf5e018 in login_binary_run () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #5 0x0000000000402459 in ?? () No symbol table info available. #6 0x00007fff8a9c65f8 in ?? () No symbol table info available. #7 0x000000000000001c in ?? () No symbol table info available. #8 0x0000000000000002 in ?? () No symbol table info available. #9 0x00007fff8a9c7e6a in ?? () No symbol table info available. #10 0x00007fff8a9c7e7d in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Sun Sep 23 16:51:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 09:51:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505F13DB.1060400@Media-Brokers.com> On 2012-09-23 2:00 AM, Joe Auty wrote: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK Please don't make it harder for others to help you... Just paste the output in the body of the email. Some (many?) people won't click on links like that from people they don't know... -- Best regards, Charles From stephan at rename-it.nl Sun Sep 23 20:31:27 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:31:27 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <20120920081126.0627cd71@tomh> References: <20120920081126.0627cd71@tomh> Message-ID: <505F476F.5090305@rename-it.nl> On 9/20/2012 2:11 PM, Tom Horsley wrote: > A slightly off topic question, but maybe someone here has seen > something like this: > > I want a program that operates entirely on the client side of > an IMAP server, and implements the sieve filtering language > by doing the necessary client side operations to move mail > on the IMAP server, expunge mail, etc. > > It would also act as a proxy between my real mail client and > the IMAP server so I'd never see any mail that hasn't been > filtered first. > > The benefits are obvious: I could finally get sieve filtering > on my stupid corporate exchange mail server :-). Interesting idea. I've been experimenting a bit with the sieve-filter command line tool and the Dovecot imapc mail storage backend. This does not entirely match your proposed solution, since it'll have to operate as a cron job, but at least it is a start. I've configured my experiment as follows: => I built a simple dovecot.conf protocols = none listen = *, :: mail_location = imapc:~/imapc imapc_host = your.host.tld imapc_port = 143 imapc_user = username imapc_password = imapc_ssl = starttls imapc_ssl_verify = no => I tested whether IMAP connection works: stephan at klara:~/test$ /usr/lib/dovecot/imap -c ./dovecot.conf * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in as stephan 2342 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8274 EXISTS * 0 RECENT * OK [UIDVALIDITY 1348169448] UIDs valid * OK [UIDNEXT 8275] Predicted next UID * OK [NOMODSEQ] No permanent modsequences 2342 OK [READ-WRITE] Select completed. => I built a Sieve script called test.sieve: require "fileinto"; require "mailbox"; # Just moves mail elsewhere fileinto :create "Target"; => And finally tested operation using sieve-filter: sieve-filter -v -e -W -c dovecot.conf test.sieve INBOX In the process a few problems were encountered and solved. This means that this little experiment will only work with the latest revisions of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective Mercurial repositories. The conclusion so far is that, in essence, what you'd like to do is possible. However, to make Sieve filtering look like it is being run at delivery rather than at regular intervals in the background, we will need to build some sort of IMAP plugin that monitors INBOX for new messages and passes them through a Sieve filter before the user's proxied IMAP client can see them. I'll have to investigate how much work implementing this would be. Regards, Stephan. From stephan at rename-it.nl Sun Sep 23 20:43:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:43:41 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <505F476F.5090305@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> Message-ID: <505F4A4D.6060304@rename-it.nl> On 9/23/2012 7:31 PM, Stephan Bosch wrote: > On 9/20/2012 2:11 PM, Tom Horsley wrote: >> I want a program that operates entirely on the client side of >> an IMAP server, and implements the sieve filtering language >> by doing the necessary client side operations to move mail >> on the IMAP server, expunge mail, etc. >> > Interesting idea. I've been experimenting a bit with the sieve-filter > command line tool and the Dovecot imapc mail storage backend. This > does not entirely match your proposed solution, since it'll have to > operate as a cron job, but at least it is a start. [...] > In the process a few problems were encountered and solved. This means > that this little experiment will only work with the latest revisions > of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective > Mercurial repositories. > > The conclusion so far is that, in essence, what you'd like to do is > possible. However, to make Sieve filtering look like it is being run > at delivery rather than at regular intervals in the background, we > will need to build some sort of IMAP plugin that monitors INBOX for > new messages and passes them through a Sieve filter before the user's > proxied IMAP client can see them. I'll have to investigate how much > work implementing this would be. I should explain explain in a little more detail how this works. Basically, I use the sieve-filter tool to filter mail that is already in a mailbox. Only, the mailbox is a bit special: it is an imapc (IMAP Client) mailbox that proxies to a remote mailbox. So, on our local machine the INBOX actually maps to INBOX on the remote server, rather than some local store. For the sieve-filter tool this makes no difference. This way, the sieve-filter can also be used to filter mail on a remote server. Regards, Stephan. From me at junc.org Sun Sep 23 20:47:17 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 23 Sep 2012 19:47:17 +0200 Subject: [Dovecot] =?utf-8?q?Sieve_as_mail_client=3F?= In-Reply-To: <505F4A4D.6060304@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> Message-ID: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Stephan Bosch skrev den 23-09-2012 19:43: > This way, the sieve-filter can also be used to filter mail > on a remote server. is it gpl ?, anyway pretty cool From stephan at rename-it.nl Sun Sep 23 21:44:36 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 20:44:36 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Message-ID: <505F5894.8040809@rename-it.nl> On 9/23/2012 7:47 PM, Benny Pedersen wrote: > Stephan Bosch skrev den 23-09-2012 19:43: >> This way, the sieve-filter can also be used to filter mail >> on a remote server. > > is it gpl ? The sieve-filter tool is part of Pigeonhole. Both Dovecot and Pigeonhole are mostly LGPLv2.1. Check the COPYING files for details. Regards, Stephan. From joe at netmusician.org Sun Sep 23 22:45:33 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 15:45:33 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F13DB.1060400@Media-Brokers.com> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> Message-ID: <505F66DD.70603@netmusician.org> > > Charles Marcus > September 23, 2012 9:51 AM > > > Please don't make it harder for others to help you... > > Just paste the output in the body of the email. Some (many?) people > won't click on links like that from people they don't know... My apologies, I figured at the time that it would be better to not make these emails crazy long, but I guess they can be just as easily snipped... Here is my conf: ## Dovecot configuration file # If you're in a hurry, see http://wiki2.dovecot.org/QuickConfiguration # "doveconf -n" command gives a clean output of the changed settings. Use it # instead of copy&pasting files when posting to the Dovecot mailing list. # '#' character and everything after it is treated as comments. Extra spaces # and tabs are ignored. If you want to use either of these explicitly, put the # value inside quotes, eg.: key = "# char and trailing whitespace " # Default values are shown for each setting, it's not required to uncomment # those. These are exceptions to this though: No sections (e.g. namespace {}) # or plugin settings are added by default, they're listed only as examples. # Paths are also just examples with the real defaults being based on configure # options. The paths listed here are for configure --prefix=/usr # --sysconfdir=/etc --localstatedir=/var # Enable installed protocols !include_try /usr/share/dovecot/protocols.d/*.protocol # A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, :: # Base directory where to store runtime data. #base_dir = /var/run/dovecot/ # Name of this instance. In multi-instance setup doveadm and other commands # can use -i to select which instance is used (an alternative # to -c ). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot # Greeting message for clients. #login_greeting = Dovecot ready. # Space separated list of trusted network ranges. Connections from these # IPs are allowed to override their IP addresses and ports (for logging and # for authentication checks). disable_plaintext_auth is also ignored for # these networks. Typically you'd specify your IMAP proxy servers here. #login_trusted_networks = # Sepace separated list of login access check sockets (e.g. tcpwrap) #login_access_sockets = # With proxy_maybe=yes if proxy destination matches any of these IPs, don't do # proxying. This isn't necessary normally, but may be useful if the destination # IP is e.g. a load balancer's IP. #auth_proxy_self = # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). #verbose_proctitle = no verbose_proctitle = yes # Should all processes be killed when Dovecot master process shuts down. # Setting this to "no" means that Dovecot can be upgraded without # forcing existing client connections to close (although that could also be # a problem if the upgrade is e.g. because of a security fix). #shutdown_clients = yes # If non-zero, run mail commands via this many connections to doveadm server, # instead of running them directly in the same process. #doveadm_worker_count = 0 # UNIX socket or host:port used for connecting to doveadm server #doveadm_socket_path = doveadm-server # Space separated list of environment variables that are preserved on Dovecot # startup and passed down to all of its child processes. You can also give # key=value pairs to always set specific settings. #import_environment = TZ ## ## Dictionary server settings ## # Dictionary can be used to store key=value lists. This is used by several # plugins. The dictionary can be accessed either directly or though a # dictionary server. The following dict block maps dictionary names to URIs # when the server is used. These can then be referenced using URIs in format # "proxy::". dict { #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext } # Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf # A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf > > > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Mon Sep 24 01:42:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 18:42:09 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F66DD.70603@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> <505F66DD.70603@netmusician.org> Message-ID: <505F9041.6080700@Media-Brokers.com> On 2012-09-23 3:45 PM, Joe Auty wrote: >> >> Charles Marcus >> September 23, 2012 9:51 AM >> >> >> Please don't make it harder for others to help you... >> >> Just paste the output in the body of the email. Some (many?) people >> won't click on links like that from people they don't know... > > My apologies, I figured at the time that it would be better to not > make these emails crazy long, but I guess they can be just as easily > snipped... Here is my conf: > > ## Dovecot configuration file No, you got it right the second time... When I said 'output', I mean the output of doveconf -n... which you did in a follow up email... -- Best regards, Charles From amateo at um.es Mon Sep 24 09:17:11 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 24 Sep 2012 08:17:11 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505FFAE7.3070905@um.es> El 21/09/12 11:32, Tom?? Randa escribi?: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. Could it be > some dovecot bug or dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap > clients in one process, but still I am over 1000 processes with kernel > message: > > maxproc limit exceeded by uid 89 > > > Could anybody help? Many thanks Tomas > Hi, I don't know BSD, but we had a similar problems with linux, when we reached 1024 processes, no more processes were created and we had errors like "imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument". If this is your same case, you could look for more info at http://www.dovecot.org/list/dovecot/2012-July/067014.html -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From manu at netbsd.org Mon Sep 24 10:39:54 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Mon, 24 Sep 2012 07:39:54 +0000 Subject: [Dovecot] dsync Message-ID: <20120924073954.GB16335@homeworld.netbsd.org> Hi Testing dsync, things go wrong: doveadm sync -u user remote:root at mail2.example.net dsync-local(user): Error: Mailboxes don't have unique GUIDs: 72e3be2c6f203b50883c000044af56a8 is shared by RT and RT_72e3be2c6f203b50883c000044af56a8 Obviously RT_72e3be2c6f203b50883c000044af56a8 is an outdated copy of RT But .mailboxlist does not list that mailbox. Is there a trick to make sure dsync only use valid mailboxes? I have this in dovecot.conf mail_location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=/mail/indexes/%u:SUBSCRIPTI ONS=../.mailboxlist Another problem, that may or may not be related: dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 60298748 dsync-local(user): Error: Failed to sync mailbox RT: Timeout while waiting for lock dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 63587421 dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files I also get this: dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files dsync-local(user): Error: proxy client timed out (waiting for MSG-GET message from remote) And this: dsync-local(user): Error: read() from worker server failed: EOF And generally speaking ,how good is dsync? is it usabel in production? This is on dovecot 2.1.7 -- Emmanuel Dreyfus manu at netbsd.org From dovecot at lists.wgwh.ch Mon Sep 24 11:49:22 2012 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Mon, 24 Sep 2012 10:49:22 +0200 Subject: [Dovecot] 2.1.10 imapc assert crash report Message-ID: <20120924104922.211d9ca3@boscos> Hi Timo I have a simple imapc gmail proxy test setup which works fine on 2.1.9, but crashes on 2.1.10 # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.4.7-1-ARCH x86_64 auth_mechanisms = plain login imapc_host = imap.gmail.com imapc_port = 993 imapc_ssl = imaps imapc_ssl_ca_dir = /etc/ssl/certs listen = 127.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=imap.gmail.com port=993 ssl=imaps default_fields = userdb_imapc_user=%u userdb_imapc_password=%w userdb_imapc_ssl=imaps userdb_imapc_port=993 driver = imap } protocols = imap ssl = no userdb { driver = prefetch } Log: Sep 24 10:21:58 codemonkey dovecot: master: Dovecot v2.1.10 starting up Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Sep 24 10:22:12 codemonkey dovecot: auth: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x453aa) [0x7f8d5ce963aa] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x453ee) [0x7f8d5ce963ee] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8d5ce6abd3] -> /usr/local/lib/dovecot/auth/libauthdb_imap.so(+0x977c) [0x7f8d5be3677c] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3618e) [0x7f8d5ce8718e] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f8d5cea3006] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f8d5cea3df7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f8d5cea2b48] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8d5ce8eb93] -> dovecot/auth(main+0x2ff) [0x40ad4f] -> /lib/libc.so.6(__libc_start_main+0xf5) [0x7f8d5c27d725] -> dovecot/auth() [0x40af61] Sep 24 10:22:12 codemonkey dovecot: auth: Fatal: master: service(auth): child 24008 killed with signal 6 (core not dumped) Sep 24 10:22:12 codemonkey dovecot: imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=24007, EOF) Oli -- message transmitted on 100% recycled electrons From tss at iki.fi Mon Sep 24 14:11:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:11:24 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: On 21.9.2012, at 8.28, Timo Sirainen wrote: >> Timo Sirainen wrote: >>> doveadm multi [-A | -u wildcards] [ [...]] >>> >>> Thoughts? > >> As command name I could also think of "doveadm sequence", which >> implies the commands being executed in serial order. > > Hmm. Maybe. "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. From tss at iki.fi Mon Sep 24 14:17:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:17:16 +0300 Subject: [Dovecot] 2.1.10 imapc assert crash report In-Reply-To: <20120924104922.211d9ca3@boscos> References: <20120924104922.211d9ca3@boscos> Message-ID: <0734901B-7805-42C5-9039-F0D6C2EC566B@iki.fi> On 24.9.2012, at 11.49, Oli Schacher wrote: > I have a simple imapc gmail proxy test setup which works fine on 2.1.9, > but crashes on 2.1.10 > Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/fd863826c892 http://hg.dovecot.org/dovecot-2.1/rev/17a8f15beb8c From Ralf.Hildebrandt at charite.de Mon Sep 24 14:27:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:27:06 +0200 Subject: [Dovecot] Logging question regarding delete actions Message-ID: <20120924112706.GS4043@charite.de> A user is logged in via imap from multiple devices. The log has this: Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 How can I find out WHICH CLIENT caused the deletion? Same issue: Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 How can I be sure that the log entry from 10:06:17 "belongs" to the log entry from "09:36:05"? Also, what is the meaning of the "session="? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:36:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:36:45 +0300 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <20120924112706.GS4043@charite.de> References: <20120924112706.GS4043@charite.de> Message-ID: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> On 24.9.2012, at 14.27, Ralf Hildebrandt wrote: > A user is logged in via imap from multiple devices. > The log has this: > > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 > > How can I find out WHICH CLIENT caused the deletion? Change mail_log_prefix to include %{session} (and maybe %r for IP). > Same issue: > > Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= > Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 > > How can I be sure that the log entry from 10:06:17 "belongs" to the > log entry from "09:36:05"? Also, what is the meaning of the > "session="? This is also solved with mail_log_prefix change. The session's idea is exactly to match the same session's log messages together. It's a string guaranteed to be unique for the next .. was it 7 years or so. From alec at alec.pl Mon Sep 24 14:44:29 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 24 Sep 2012 13:44:29 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> Message-ID: <5060479D.3060609@alec.pl> On 09/22/2012 06:50 PM, Timo Sirainen wrote: > On 21.9.2012, at 11.23, A.L.E.C wrote: > >> On 09/20/2012 06:01 PM, Timo Sirainen wrote: >>> Thoughts? Any better name for the command than "multi"? >> >> How about 'execute' or 'exec'. > > v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. next is "run" or "pipe", but what if you create global separator option and detect multi-command syntax usage automatically without a keyword? Syntax for doveadm would be doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command [command_options] [command_arguments] [separator command [command_options] [command_arguments] [...]] and example doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Ralf.Hildebrandt at charite.de Mon Sep 24 14:56:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:56:45 +0200 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> References: <20120924112706.GS4043@charite.de> <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> Message-ID: <20120924115645.GY4043@charite.de> * Timo Sirainen : > This is also solved with mail_log_prefix change. The session's idea is > exactly to match the same session's log messages together. It's a > string guaranteed to be unique for the next .. was it 7 years or so. Thanks. I changed the mail_log_prefix from mail_log_prefix = "%s(%u): " to mail_log_prefix = "%s(%u) %{session}: " -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:58:53 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5060479D.3060609@alec.pl> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> Message-ID: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> On 24.9.2012, at 14.44, A.L.E.C wrote: > next is "run" or "pipe", but what if you create global separator option > and detect multi-command syntax usage automatically without a keyword? > > Syntax for doveadm would be > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > [command_options] [command_arguments] [separator command > [command_options] [command_arguments] [...]] > > and example > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge Hmm. Yes, that might work. Although it would have to be: doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge because both -A and -s are mail command specific parameters, which won't work for non-mail commands. Hmm. This reminds me also that it would be possible with some extra work to do some command interaction. IMAP supports saving search results, which can later be accessed with $ parameter. So this could be made to work: doveadm search -s : from foo : fetch text \$ : expunge \$ From chevalier at dspnet.fr Mon Sep 24 16:48:11 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 15:48:11 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) Message-ID: <20120924134810.GA62723@kyoko.org> Hello, I don't know if it's been addressed before, but anyway : In my dovecot setup, I have local and virtual users. So, I need multiple passdb backends. Namely, passwd for the local users and ldap for the virtual users. passdb { driver = passwd } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } Everything work correctly : when a user logs in (imap/pop3) there's a lookup in passwd and if it fails there's a lookup in ldap (if I understand the process correctly), which eventually succeeds. Except that every time a virtual user logs in, dovecot logs an error, like : dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument I guess it's because the login is a full email that getpwnam fails. Anyway, the user logs in just fine. But I would like to know if/how I can get rid of the messages filling my logs ? I tried : auth_debug_passwords = no auth_verbose = no But no dice. I used dovecot 1.x before and there was no such messages. Thanks for any advice. K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From tss at iki.fi Mon Sep 24 17:16:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:16:06 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924134810.GA62723@kyoko.org> References: <20120924134810.GA62723@kyoko.org> Message-ID: On 24.9.2012, at 16.48, Philippe Chevalier wrote: > dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument > > I guess it's because the login is a full email that getpwnam fails. So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 386 bytes Desc: not available URL: From tss at iki.fi Mon Sep 24 17:32:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:32:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EED00.6090109@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: On 23.9.2012, at 14.05, Joe Auty wrote: > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? From blevi.linux at gmail.com Mon Sep 24 17:55:18 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 17:55:18 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <50607456.1040709@gmail.com> On 24/09/2012 17:32, Timo Sirainen wrote: > On 23.9.2012, at 14.05, Joe Auty wrote: > >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 > > Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Levi From tss at iki.fi Mon Sep 24 17:58:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:58:17 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <50607456.1040709@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> Message-ID: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> On 24.9.2012, at 17.55, Birta Levente wrote: > On 24/09/2012 17:32, Timo Sirainen wrote: >> On 23.9.2012, at 14.05, Joe Auty wrote: >> >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> (gdb) bt full >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> >> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >> > > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? From blevi.linux at gmail.com Mon Sep 24 18:04:29 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 18:04:29 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <5060767D.50002@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > I build my own rpm based on src rpm: dovecot-2.1.1-2_132.src.rpm. #dovecot -n auth_mechanisms = plain login cram-md5 debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no listen = * mail_access_groups = vmail mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = quota mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Quota exceeded, please contact postmaster at benvenuti.ro quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=85%% quota-warning 85 %u quota_warning4 = storage=80%% quota-warning 80 %u quota_warning5 = storage=50%% quota-warning 50 %u } postmaster_address = postmaster at mydomain.com service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0640 user = vmail } user = vmail } ssl_ca = References: <20120924134810.GA62723@kyoko.org> Message-ID: <20120924150440.GA85969@kyoko.org> On Mon, Sep 24, 2012 at 05:16:06PM +0300, Timo Sirainen wrote: >On 24.9.2012, at 16.48, Philippe Chevalier wrote: > >> dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument >> >> I guess it's because the login is a full email that getpwnam fails. > >So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? If I log in as a non-existent user (neither in passwd or ldap), without the domain part, it logs also an error, but this time from ldap : dovecot: auth: Error: ldap(foo.bar,xx.xx.xx.xx,): ldap_bind() failed: Invalid DN syntax My bind DN to check the password is : auth_bind_userdn = dc=%n,dc=%d,ou=Domains,ou=Mail,dc=dspnet,dc=fr (I have virtual users in multiple domains) So ldap protests probably because the "domain" part is missing. If I use a non-existent login "foo at bar", dovecot logs nothing : no error from passwd, no error from ldap, just an authentication error on the client side. I will apply the patch later today and will let you know the result. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From alessio at skye.it Mon Sep 24 18:40:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 17:40:52 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <50607F04.9040104@skye.it> Il 19/09/2012 15:07, Alessio Cecchi ha scritto: > Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>> >>>> LDA is configured and works fine but the problem is when the first >>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>> message is written to the user's Mailbox but the message remains, >>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>> the second attempt is delivered fine. >>> gdb backtrace would be very helpful in figuring out the problem: >>> http://dovecot.org/bugreport.html >>> >> Hi Timo, had you occasion to see the problem? Can I provide more information? Thanks > > This is the full bt: > > (gdb) bt full > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > ns = > ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = > 0x0, > v_modifiable = 0x0} > ids = 0x1928658 > i = > dest = > ret = -883075307 > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 > auser = 0x1949a08 > iter = 0x0 > acllist_path = 0x1928658 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" > ret = > ns = 0x1943e50 > output = 0x0 > st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = > 33152, > st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, > st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = > 1348059559, > tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, > st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, > 0, 0}} > path = 0x1928210 > > file_mode = 384 > dir_mode = 448 > gid = 4294967295 > list = > info = > rootdir = 0x1928610 "Sent" > origin = 0x194d178 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" > fd = 8 > #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) > at acl-backend-vfile-acllist.c:311 > acllist_path = > #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh > (backend=0x1944240) > at acl-backend-vfile-acllist.c:153 > __FUNCTION__ = "acl_backend_vfile_acllist_refresh" > #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, > name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 > acllist = > #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( > _aclobj=0x19444e0) at acl-backend-vfile.c:858 > old_validity = > validity = {global_validity = {last_check = 0, > last_read_time = 1348059559, last_mtime = 0, last_size = 0}, > local_validity = {last_check = 0, last_read_time = 0, > > last_mtime = 0, last_size = 0}, mailbox_validity = { > last_check = 0, last_read_time = 0, last_mtime = 0, > last_size = 0}} > mtime = 0 > ret = 26515976 > #6 0x00007f2fc9fc125e in acl_backend_get_default_rights > (backend=0x1944240, > mask_r=0x28) at acl-backend.c:164 > No locals. > #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at acl-mailbox-list.c:107 > alist = > nonowner_list_ctx = > ret = > backend = 0x1944240 > acl_mask = 0x1 > ns = 0x1943e50 > update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = > 0x7f2fcbf2ba88, > glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, > update_only = 0, match_parents = 0} > name = > #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, > flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 > _data_stack_cur_id = 2 > > ctx = 0x1946b20 > pool = > i = > inboxcase = > #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple > (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at mailbox-list-iter.c:158 > ctx = > ret = > __FUNCTION__ = "mailbox_list_iter_init_multiple" > #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, > pattern=, flags=1348059559) at > mailbox-list-iter.c:58 > patterns = {0x7f2fc9db76dc "*", 0x0} > #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, > bytes_r=, count_r=0x7fff362dfff0) at > quota-count.c:73 > ctx = 0x7f2fcb5beef3 > info = > #12 quota_count (root=0x1944950, bytes_r=, > count_r=0x7fff362dfff0) at quota-count.c:111 > i = 0 > ret = 0 > #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, > value_r=0x7fff362e0038) at quota-dict.c:113 > > dt = > bytes = 0 > count = 0 > #14 0x00007f2fc9db395a in dict_quota_update_callback ( > ret=, context=0x1949a08) at quota-dict.c:178 > value = 1 > #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, > line_r=) at dict-client.c:265 > ctx = 0x1984190 > #16 client_dict_read_one_line (dict=0x194c7f0, line_r= out>) > at dict-client.c:356 > id = 1 > line = > ret = 0 > __FUNCTION__ = "client_dict_read_one_line" > #17 0x00007f2fcb592565 in client_dict_wait (_dict=) > at dict-client.c:520 > dict = 0x194c7f0 > line = 0x0 > ret = > #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) > at quota-dict.c:90 > root = 0x1944950 > > #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 > pool = 0x194c5c0 > #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 > quota = 0x1944720 > i = 2 > #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) > at quota-storage.c:412 > quser = 0x1949a30 > quota_set = 0x1946f30 > #22 0x00007f2fcb88188e in mail_user_unref (_user=) > at mail-user.c:153 > user = 0x19483c0 > __FUNCTION__ = "mail_user_unref" > #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 > set_roots = {0x604640, 0x0} > ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, > dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, > src_envelope_sender = 0x0, dest_user = 0x0, > dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, > var_expand_table = 0x192e050, tried_default_save = true, > saved_mail = true, save_dest_mail = false, mailbox_full = > false, > > dsn = false} > service_flags = > user = 0x192d3c2 "cecchi10 at qboxdns.it" > errstr = 0x0 > path = 0x7fff362e0368 "\351\a" > storage_service = 0x192f3a0 > service_user = 0x192fd48 > service_input = {module = 0x4034d5 "lda", service = 0x4034d5 > "lda", > username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, > local_ip = {family = 0, u = {ip6 = {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = > {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, local_port = 0, remote_port = 0, > userdb_fields = 0x0, flags_override_add = 0, > flags_override_remove = 0, no_userdb_lookup = 0} > storage = 0x1943f30 > user_source = > destaddr_source = 0x403594 "user at hostname" > process_euid = > stderr_rejection = false > > ret = > c = > error = MAIL_ERROR_NONE > (gdb) > > Sorry > -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From feltrin at gmail.com Mon Sep 24 18:44:52 2012 From: feltrin at gmail.com (Jean Michel) Date: Mon, 24 Sep 2012 12:44:52 -0300 Subject: [Dovecot] Patches and dovecot releases Message-ID: I'd like to know if after a release, for example the recently release 2.1.10, its common to see few days from the release some bug reports and algo some patches, are theses patches applyed on the daily builds ? ------------------------------ Jean Michel Feltrin From m.naumann at globalview.de Mon Sep 24 19:32:43 2012 From: m.naumann at globalview.de (M. Naumann) Date: Mon, 24 Sep 2012 18:32:43 +0200 Subject: [Dovecot] Traffic Accounting Message-ID: <50608B2B.5050207@globalview.de> Hi, I'm trying to find out how to do traffic accounting with Dovecot 2.x, preferrably v2.0.9, preferrably on CentOS 6. I've previously asked on IRC, but there was little feedback, and my understanding is now this list is the preferred media for such inquiries. If I recall correctly, some weeks ago I was told that traffic accounting is not officially supported on Dovecot 2, but that there could still be ways to get it to work, but no details were provided. I can think of the following approaches: * rawlog, preferrably piped (if that's possible?) into something like wc to prevent privacy issues and to reduce the I/O overhead * maildrop filtering in front of dovecot LDA (for mail inbound to mail storage) * sieve filtering Unfortunately I have little experience with either so far, so it's hard to make a good choice. I would appreciate hints on these approaches, and on any other approaches you can think of, as well as any related documentation / how-to you could point me to. While I'm subscribed to the list (for mail authentication purposes), I've disabled receiving any e-mail form the list, so please CC me on any replies. Thanks in advance, Moritz From robert at schetterer.org Mon Sep 24 19:46:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:46:23 +0200 Subject: [Dovecot] Patches and dovecot releases In-Reply-To: References: Message-ID: <50608E5F.2080704@schetterer.org> Am 24.09.2012 17:44, schrieb Jean Michel: > I'd like to know if after a release, for example the recently release > 2.1.10, its common to see few days from the release some bug reports and > algo some patches, are theses patches applyed on the daily builds ? > > > ------------------------------ > Jean Michel Feltrin > look http://hg.dovecot.org/ dovecot has tons of config options, and there are extrem multi ways to set it up ( for this we love dovecot ), so some stuff may bug in a stable release for some setups/functions sometimes also distros do self patching, there are sometimes kernel bugs etc in general its not a software having bugs, its the way and speed they get fixed, and Timo does a "wonderman" job here after all ,unless there are no security fixes, or you urgent need new setup options etc, nobody presses you to upgrade in a stable tree read the changelogs -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Sep 24 19:48:38 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:48:38 +0200 Subject: [Dovecot] Traffic Accounting In-Reply-To: <50608B2B.5050207@globalview.de> References: <50608B2B.5050207@globalview.de> Message-ID: <50608EE6.6030905@schetterer.org> Am 24.09.2012 18:32, schrieb M. Naumann: > Hi, > > I'm trying to find out how to do traffic accounting with Dovecot 2.x, > preferrably v2.0.9, preferrably on CentOS 6. > > I've previously asked on IRC, but there was little feedback, and my > understanding is now this list is the preferred media for such > inquiries. If I recall correctly, some weeks ago I was told that traffic > accounting is not officially supported on Dovecot 2, but that there > could still be ways to get it to work, but no details were provided. > > I can think of the following approaches: > > * rawlog, preferrably piped (if that's possible?) into something like wc > to prevent privacy issues and to reduce the I/O overhead > > * maildrop filtering in front of dovecot LDA (for mail inbound to mail > storage) > > * sieve filtering > > Unfortunately I have little experience with either so far, so it's hard > to make a good choice. I would appreciate hints on these approaches, and > on any other approaches you can think of, as well as any related > documentation / how-to you could point me to. > > While I'm subscribed to the list (for mail authentication purposes), > I've disabled receiving any e-mail form the list, so please CC me on any > replies. > > Thanks in advance, > > Moritz > perhaps this helps on the 2.1.x tree there is a stats plugin http://wiki2.dovecot.org/Statistics and/or use syslog analysers etc -- Best Regards MfG Robert Schetterer From ben at morrow.me.uk Mon Sep 24 20:01:02 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:01:02 +0100 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> Message-ID: <20120924170101.GB63656@anubis.morrow.me.uk> At 2PM +0300 on 24/09/12 you (Timo Sirainen) wrote: > On 24.9.2012, at 14.44, A.L.E.C wrote: > > > next is "run" or "pipe", but what if you create global separator option > > and detect multi-command syntax usage automatically without a keyword? > > > > Syntax for doveadm would be > > > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > > [command_options] [command_arguments] [separator command > > [command_options] [command_arguments] [...]] > > > > and example > > > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge > > Hmm. Yes, that might work. Although it would have to be: > > doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge > > because both -A and -s are mail command specific parameters, which > won't work for non-mail commands. > > Hmm. This reminds me also that it would be possible with some extra > work to do some command interaction. IMAP supports saving search > results, which can later be accessed with $ parameter. So this could > be made to work: > > doveadm search -s : from foo : fetch text \$ : expunge \$ This is turning into a proper scripting language, so perhaps something like doveadm -e 'search from foo; fetch text $; expunge $' with 'doveadm -F file' to run a script file? Ben From ben at morrow.me.uk Mon Sep 24 20:11:23 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:11:23 +0100 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505FFAE7.3070905@um.es> References: <505C3435.30601@blueboard.cz> <505FFAE7.3070905@um.es> Message-ID: <20120924171123.GC63656@anubis.morrow.me.uk> > El 21/09/12 11:32, Tom?? Randa escribi?: > > Hello, > > > > I still cannot get dovecot running with more then 1000 processes, but > > hard limit is 8192 per user in box. I tried everything, including > > modifying startup script of dovecot to set ulimit -u 8192. Could it be > > some dovecot bug or dovecot<>freebsd bug? > > I also tried to set client_limit=2 in imap service to spawn more imap > > clients in one process, but still I am over 1000 processes with kernel > > message: > > > > maxproc limit exceeded by uid 89 You may be running into the kern.maxprocperuid sysctl setting. This is initialized to 9/10ths of kern.maxproc, but can be changed independantly. If you do this you may want to consider setting a default maxproc rlimit in login.conf for the other users on the box. (You may, of course, already have a maxproc limit in login.conf, and that's what's causing the problem, though the default install doesn't include one.) If you have procfs mounted you can check the maxproc rlimit of a running process by looking in /proc/pid/rlimit. In principle it's possible to also get this information with libkvm, but it's not very easy and I don't think any of the standard utilities expose it. Ben From alessio at skye.it Mon Sep 24 20:19:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 19:19:30 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <50607F04.9040104@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <50607F04.9040104@skye.it> Message-ID: <50609622.7060702@skye.it> Il 24/09/2012 17:40, Alessio Cecchi ha scritto: > Il 19/09/2012 15:07, Alessio Cecchi ha scritto: >> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >>> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>>> >>>>> LDA is configured and works fine but the problem is when the first >>>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>>> message is written to the user's Mailbox but the message remains, >>>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>>> the second attempt is delivered fine. >>>> gdb backtrace would be very helpful in figuring out the problem: >>>> http://dovecot.org/bugreport.html >>>> >>> > > Hi Timo, > > had you occasion to see the problem? Can I provide more information? > > Thanks After further testing I found this behavior, a note, I'm using dict quota in mysql. - add a new user - delivery the first email via deliver - "Segmentation fault" - I remove the newly created user - add the same user - delivery the first email via deliver - OK - add a new user - the user connects via pop/imap - delivery the first email via deliver - OK - add a new user - manually create the entry for dict quota <<== - delivery the first email via deliver - OK it seems that if there is user's entry in the dict database the problem does not appear. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stsiol at yahoo.co.uk Mon Sep 24 20:42:35 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 18:42:35 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb Message-ID: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, Xmail and OpenLDAP for a company and serving about 40 acouunts. The machine is wonderful. I am very happy with it. However, I am running out of disk space. It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity has reached 82%.? I am starting of getting nervous. Does anyone know of a painless way to migrate the entire contents directly to another pair of 146Gb SCSI RAID1 disks ? I thought of downtime and using clonezilla, but my last experience with it was questionable. I remember having problems declaring disk re-sizing from the smaller capacity drives to the larger ones. CentOS 5.5 Manual install of : Mysql XMail (pop3/smtp) ASSP (anti spam) Apache / LAMP and last but by no means list : Dovecot Dovecot -n : # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 96 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd Any help would be appreciated or any ideas you might have. Regards, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From andrei.michescu at miau.ca Mon Sep 24 21:01:33 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:01:33 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Hello Spyros, As "best practice" you never have the OS and the data/logs/user homes on the same partition or set of disks. If this is the case then your life is pretty easy: -simply create the new set of partitions -mount the new ones in a temporary location -rsync (or copy everything from old partitions) -Stop dovecot / all other daemons that might be using the data -mount the new ones in the place of old ones, mount the old ones in the place of new ones - rsync again (should be quick as not many things changed) - start all your deamons again :P If you do not have separate partitions maybe this is the perfect time to look into that... I would also look into btrfs... might be a good pick for your new partitions. best regards, Andrei > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%.? > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > ? verbose: yes > ? debug: yes > ? debug_passwords: yes > ? passdb: > ? ? driver: passwd-file > ? ? args: /etc/dovecot/passwd > ? passdb: > ? ? driver: pam > ? userdb: > ? ? driver: static > ? ? args: uid=vmail gid=vmail home=/home/vmail/%u > ? userdb: > ? ? driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis? > > !DSPAM:50609d2c301831828332458! > > From robert at schetterer.org Mon Sep 24 21:06:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:06:18 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A11A.9000007@schetterer.org> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > rsync should do the job depending on your whole machine setup it might only be only umount old /home and mount new(bigger) /home after sync ,perhaps with tmp store elsewhere ( for sure you have to have a plan before doing..) but your dovecot is very outdated, i would recommend get up to new hard and software/os install, and then migrate to new machine > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From lists at wildgooses.com Mon Sep 24 21:07:00 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:07:00 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A144.9040607@wildgooses.com> This is one of those questions which is almost too easy if you are familiar with Linux. Trying not to sound like a d*ck, but is it an option to rent someone to help with admin jobs? For example, were it me then I would probably have setup some partitioning scheme with separate partitions for data and operating system? Possibly also using LVM? You have several options, mainly the choice of filesystem will dictate here, but quite possibly you can: 1) Pull the drives one by one and rebuild the raid after each. Keep the old drives since you can technically roll back onto them. Expand the partitions (scary without LVM) and then expand the filesystem on the partitions 2) Boot from a DVD/Flash on your favourite rescue distro (I like sysrecuecd). Create the new raid, copy the old to the new, remove the old drives, reboot from new. Possibly taking the time to repartition and move some data around while you do it (remember to update fstab) Both are fairly simple if you have done it once, but it would be well worth finding someone either local or who will log in via remote control and support you? Final thought: For the size of drives you are looking at, SSD drives are relatively inexpensive and likely comparable with the high end drives you are probably looking to buy? For 40 users I would hazard a guess you likely would be happy with inexpensive low end drives, but certainly a couple of small SSDs will blow away a spinning disk and give you a decent upgrade... Good luck Ed W On 24/09/2012 18:42, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From andrei.michescu at miau.ca Mon Sep 24 21:07:11 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:07:11 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Message-ID: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Hello Spyros, Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA connector... so the solution below does not really apply to you :( Andrei > Hello Spyros, > > As "best practice" you never have the OS and the data/logs/user homes on > the same partition or set of disks. > > If this is the case then your life is pretty easy: > -simply create the new set of partitions > -mount the new ones in a temporary location > -rsync (or copy everything from old partitions) > -Stop dovecot / all other daemons that might be using the data > -mount the new ones in the place of old ones, mount the old ones in the > place of new ones > - rsync again (should be quick as not many things changed) > - start all your deamons again :P > > If you do not have separate partitions maybe this is the perfect time to > look into that... > > I would also look into btrfs... might be a good pick for your new > partitions. > > best regards, > Andrei > >> Hello all, >> >> I have a DL360 G4 1U server that does a wonderfull job with dovecot >> horde, >> Xmail and OpenLDAP for a company and serving about 40 acouunts. >> >> The machine is wonderful. I am very happy with it. >> However, I am running out of disk space. >> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >> has reached 82%.? >> >> I am starting of getting nervous. >> >> Does anyone know of a painless way to migrate the entire contents >> directly >> to another pair of 146Gb SCSI RAID1 disks ? >> >> I thought of downtime and using clonezilla, but my last experience with >> it >> was questionable. I remember having problems declaring disk re-sizing >> from the smaller capacity drives to the larger ones. >> >> CentOS 5.5 >> Manual install of : >> >> Mysql >> XMail (pop3/smtp) >> ASSP (anti spam) >> Apache / LAMP >> and last but by no means list : Dovecot >> >> Dovecot -n : >> >> # 1.2.16: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >> base_dir: /var/run/dovecot/ >> log_path: /var/log/dovecot/dovecot.log >> info_log_path: /var/log/dovecot/dovecot-info.log >> ssl_parameters_regenerate: 48 >> verbose_ssl: yes >> login_dir: /var/run/dovecot//login >> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >> login_greeting: * Dovecot ready * >> login_max_processes_count: 96 >> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >> mail_plugins: zlib >> auth default: >> ? verbose: yes >> ? debug: yes >> ? debug_passwords: yes >> ? passdb: >> ? ? driver: passwd-file >> ? ? args: /etc/dovecot/passwd >> ? passdb: >> ? ? driver: pam >> ? userdb: >> ? ? driver: static >> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >> ? userdb: >> ? ? driver: passwd >> >> >> Any help would be appreciated or any ideas you might have. >> >> Regards, >> >> spyros >> >> >> >> >> >> >> ---- >> "I merely function as a channel that filters >> music through the chaos of noise" >> - Vangelis? >> >> >> >> > > > > !DSPAM:5060a006309197419291868! > > From robert at schetterer.org Mon Sep 24 21:23:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:23:25 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <5060A51D.80404@schetterer.org> Am 24.09.2012 20:07, schrieb Michescu Andrei: > Hello Spyros, > > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( > > Andrei depends how long downtime is acceptable i.e go rsync tmp over usb storage, or simple tmp nfs mounts to other servers are thinkable to minimize downtime should be no big problem its only one server with maildir and less mailboxes and data but thats all much off topic with dovecot -- Best Regards MfG Robert Schetterer From chevalier at dspnet.fr Mon Sep 24 21:41:57 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 20:41:57 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924150440.GA85969@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> Message-ID: <20120924184157.GA75341@kyoko.org> On Mon, Sep 24, 2012 at 05:04:40PM +0200, Philippe Chevalier wrote: > >I will apply the patch later today and will let you know the result. I applied the patch, and obviously, when getpwnam_r sets the result to NULL and returns EINVAL, dovecot do as if the entry was not found and stays mute. So, thank you, auth is now a lot less noisy. As for the ldap message, it errors if there's no domain in the login. In the doc, it says that %d is empty if there's no domain part. So I guess it's an enhancement request : a configuration option to have it filled out with a default domain if there's no one supplied by the client. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From stsiol at yahoo.co.uk Mon Sep 24 21:42:21 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 19:42:21 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> ----- Original Message ----- > From: Michescu Andrei > To: Dovecot Mailing List > Cc:? > Sent: Monday, 24 September 2012, 21:07 > Subject: Re: [Dovecot] 76Gb to 146Gb >? > Hello Spyros, >? > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( >? > Andrei >? >> Hello Spyros, >>? >> As "best practice" you never have the OS and the data/logs/user? > homes on >> the same partition or set of disks. >>? >> If this is the case then your life is pretty easy: >> ? -simply create the new set of partitions >> ? -mount the new ones in a temporary location >> ? -rsync (or copy everything from old partitions) >> ? -Stop dovecot / all other daemons that might be using the data >> ? -mount the new ones in the place of old ones, mount the old ones in the >> place of new ones >> ? - rsync again (should be quick as not many things changed) >> ? - start all your deamons again :P >>? >> If you do not have separate partitions maybe this is the perfect time to >> look into that... >>? >> I would also look into btrfs... might be a good pick for your new >> partitions. >>? >> best regards, >> Andrei >>? >>> Hello all, >>>? >>> I have a DL360 G4 1U server that does a wonderfull job with dovecot >>> horde, >>> Xmail and OpenLDAP for a company and serving about 40 acouunts. >>>? >>> The machine is wonderful. I am very happy with it. >>> However, I am running out of disk space. >>> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >>> has reached 82%.? >>>? >>> I am starting of getting nervous. >>>? >>> Does anyone know of a painless way to migrate the entire contents >>> directly >>> to another pair of 146Gb SCSI RAID1 disks ? >>>? >>> I thought of downtime and using clonezilla, but my last experience with >>> it >>> was questionable. I remember having problems declaring disk re-sizing >>> from the smaller capacity drives to the larger ones. >>>? >>> CentOS 5.5 >>> Manual install of : >>>? >>> Mysql >>> XMail (pop3/smtp) >>> ASSP (anti spam) >>> Apache / LAMP >>> and last but by no means list : Dovecot >>>? >>> Dovecot -n : >>>? >>> # 1.2.16: /etc/dovecot/dovecot.conf >>> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >>> base_dir: /var/run/dovecot/ >>> log_path: /var/log/dovecot/dovecot.log >>> info_log_path: /var/log/dovecot/dovecot-info.log >>> ssl_parameters_regenerate: 48 >>> verbose_ssl: yes >>> login_dir: /var/run/dovecot//login >>> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >>> login_greeting: * Dovecot ready * >>> login_max_processes_count: 96 >>> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >>> mail_plugins: zlib >>> auth default: >>> ? verbose: yes >>> ? debug: yes >>> ? debug_passwords: yes >>> ? passdb: >>> ? ? driver: passwd-file >>> ? ? args: /etc/dovecot/passwd >>> ? passdb: >>> ? ? driver: pam >>> ? userdb: >>> ? ? driver: static >>> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >>> ? userdb: >>> ? ? driver: passwd >>>? >>>? >>> Any help would be appreciated or any ideas you might have. >>>? >>> Regards, >>>? >>> spyros >>>? >>>? >>>? >>>? >>>? >>>? >>> ---- >>> "I merely function as a channel that filters >>> music through the chaos of noise" >>> - Vangelis? >>>? >>>? >>>? >>>? >>? >>? >>? >> !DSPAM:5060a006309197419291868! Andrei, Thank you very much for you kind reply and both your messages. Having said that, would it be possible to take away on 72Gb drive (say Drive1 the second drive) and shove in one of the two 146Gb ones ? Shouldn't the array be rebuilt ? Will it use the extra disk space though ? Thanks, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From lists at wildgooses.com Mon Sep 24 21:55:07 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:55:07 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A144.9040607@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> Message-ID: <5060AC8B.7090703@wildgooses.com> On 24/09/2012 19:07, Ed W wrote: > This is one of those questions which is almost too easy if you are > familiar with Linux. Trying not to sound like a d*ck, but is it an > option to rent someone to help with admin jobs? For example, were it > me then I would probably have setup some partitioning scheme with > separate partitions for data and operating system? Possibly also using > LVM? That came out wrong... What I meant to say was something more like "if you were to employ someone locally they would probably give you a whole bunch of ideas on how you could adjust the setup of the server to be more future proof. It would be worth working with someone just to get that right. For example, here are some ideas that occur to me that you could use ..." Sorry, should re-read my words before hitting send Ed From joe at netmusician.org Mon Sep 24 22:03:46 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 24 Sep 2012 15:03:46 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <5060AE92.5040904@netmusician.org> > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? Thanks Timo, I have done so. Here is the results of my debugging info now: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap-login...done. (no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 hash_table_destroy (_table=0x28) at hash.c:106 106 hash.c: No such file or directory. in hash.c (gdb) bt full #0 hash_table_destroy (_table=0x28) at hash.c:106 table = #1 0x00007ff300721054 in settings_parser_deinit (_ctx=) at settings-parser.c:237 ctx = 0x0 #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) at master-service-settings-cache.c:86 cache = 0x9f9a60 entry = 0xa016e0 next = 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x00007ff3009a5018 in main_deinit (binary=, argc=2, argv=0x9f8370) at main.c:355 No locals. #4 login_binary_run (binary=, argc=2, argv=0x9f8370) at main.c:407 set_pool = 0x9f8a30 allow_core_dumps = login_socket = c = #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #6 0x0000000000402459 in _start () No symbol table info available. (gdb) > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From stsiol at yahoo.co.uk Mon Sep 24 22:13:50 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:13:50 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060AC8B.7090703@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> <5060AC8B.7090703@wildgooses.com> Message-ID: <1348514030.72104.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: Ed W > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:55 > Subject: Re: [Dovecot] 76Gb to 146Gb > > On 24/09/2012 19:07, Ed W wrote: >> This is one of those questions which is almost too easy if you are familiar > with Linux.? Trying not to sound like a d*ck, but is it an option to rent > someone to help with admin jobs?? For example, were it me then I would probably > have setup some partitioning scheme with separate partitions for data and > operating system? Possibly also using LVM? > > That came out wrong...? What I meant to say was something more like "if you > were to employ someone locally they would probably give you a whole bunch of > ideas on how you could adjust the setup of the server to be more future proof.? > It would be worth working with someone just to get that right.? For example, > here are some ideas that occur to me that you could use ..." > > Sorry, should re-read my words before hitting send > > Ed > Ed, Don't worry about it. I wasn't offended. I have a lot of experience with linux but not on heavy metal servers. I used to have plenty of experience back in the G2/ G3 era (I was also ACE in the Compaq years) but that was back in the time that Compaq was only supporting Windows OSs and SCO. Also the problem is that I don't have the time to "play" with a spare HP/Compaq server ( I have a couple laying around btw). I'll get round to it at some point. I am just asking you chaps because I am sure people out there had the chance to tinker with newer and better equipment. Thank you for your reply, Best Regards, spyros ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Sep 24 22:24:37 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:24:37 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A11A.9000007@schetterer.org> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> Message-ID: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> ----- Original Message ----- > From: Robert Schetterer > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:06 > Subject: Re: [Dovecot] 76Gb to 146Gb > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >> Hello all, >> > ----%<----%<----%<----%<----%<----%<----%<----%<---- >? > rsync > should do the job > > depending on your whole machine setup it might only be only > umount old /home and mount new(bigger) /home after sync > ,perhaps with tmp store elsewhere > ( for sure you have to have a plan before doing..) > > but your dovecot is very outdated, i would recommend > get up to new hard and software/os install, and then migrate > to new machine > >> > > > -- > Best Regards > MfG Robert Schetterer > On client machines I have thunderbird. What if : 1. I would make sure that thunderbird keeps a local copy of all the message (I think there is a check box somewhere on settings) 2. Make sure all client machines have synced their mailboxes locally on thunderbird. 3. Install a new version of Dovecot/Horde/XMail etc. 4. When the new installation is done, try to sync from the existing clients pc's to dovecot ? Would that work ? It's one scenario I am seriously contemplating. Thank you very much again, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From me at junc.org Mon Sep 24 23:59:00 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 22:59:00 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <6ebfff9b6a3d0faeb0a91719e638e232@junc.org> Spyros Tsiolis skrev den 24-09-2012 19:42: > Any help would be appreciated or any ideas you might have. try google "centos cloud server" if you would like to do it local, use all 4 drives with 2 raid1 in the same controller if possible, then use sysrescue cd to tar it all over to the other raid1 while its down no matter how, it will be downtime i am not using centos here so i cant be more specifik http://www.sysresccd.org From me at junc.org Tue Sep 25 00:09:31 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 23:09:31 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <828720e5142c1cb59e96bcd9a15b0efb@junc.org> Spyros Tsiolis skrev den 24-09-2012 20:42: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? this can be done yes, but you will have to do more steps :) first step, remove one drive add the 146 drive wait it for rebuildin when done, remove the last small drive add the last 146 drive wait for it to rebuild now at this stage you have 72g more unused room for new partions make this new partion /home and after its being created, move the user data to it, but this leves 72g system partion with just few gigs needed ?, then i would create the new partion as lwm2, and then possible shrink system, and mount the lwm2 as /home, that way you have more options later if 146 will be to small again warn i have not doing this myself, but if should work in teori atleast From daniel.parthey at informatik.tu-chemnitz.de Tue Sep 25 00:24:48 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 24 Sep 2012 23:24:48 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: <20120924212448.GA9310@daniel.localdomain> Timo Sirainen wrote: > On 21.9.2012, at 8.28, Timo Sirainen wrote: > > >> Timo Sirainen wrote: > >>> doveadm multi [-A | -u wildcards] [ [...]] > >>> > >>> Thoughts? > > > >> As command name I could also think of "doveadm sequence", which > >> implies the commands being executed in serial order. > > > > Hmm. Maybe. > > "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. Ok, so how about "batch"? It reads a series of commands and collects them into one "batch job" which is then carried out. http://en.wikipedia.org/wiki/Batch_(Unix) Regards Daniel -- https://plus.google.com/103021802792276734820 From stan at hardwarefreak.com Tue Sep 25 01:05:35 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 24 Sep 2012 17:05:35 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <5060D92F.2070106@hardwarefreak.com> On 9/24/2012 1:42 PM, Spyros Tsiolis wrote: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? It's always best to manually take a drive off line before pulling it. > Shouldn't the array be rebuilt ? Depends on how your 6i is configured. Best guess is that it will automatically rebuild the mirror on the new 146GB drive, but... > Will it use the extra disk space though ? It will probably not. You need to read the 6i manual. I sense a hardware upgrade in your near future, either an HP server with 4 bays, or an SFF8088 JBOD chassis and an inexpensive RAID card. You already have the 146GB drives correct? They are HP pluggable drives? Which means they only work in HP gear. If that's the case you need a new server with at least 4 drive bays. You you need to buy an off brand JBOD box and two standard SATA drives. Or maybe your organizations needs more storage on many servers, and it's time to step up to an iSCSI SAN array. -- Stan From michael at orlitzky.com Tue Sep 25 05:59:15 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 22:59:15 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50611E03.3040706@orlitzky.com> On 09/24/2012 01:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. We've done this on the same hardware. You can pick up these servers for cheap; just buy an extra one. Take the new machine, throw two big disks in it, and install Gentoo. Rsync the important stuff. Make sure all of the services are working on the new machine. When you're ready to make the switch, disable external networking on the current live server. Rsync everything again, and then turn the old server off. Add its IP address to the new server. Maybe kick your router's ARP cache to expedite the change. It should only cause a minute or two of downtime. From michael at orlitzky.com Tue Sep 25 06:02:55 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 23:02:55 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <50611E03.3040706@orlitzky.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <50611E03.3040706@orlitzky.com> Message-ID: <50611EDF.2060602@orlitzky.com> On 09/24/2012 10:59 PM, Michael Orlitzky wrote: > > We've done this on the same hardware. You can pick up these servers for > cheap; just buy an extra one. Take the new machine, throw two big disks > in it, and install Gentoo. I seem to have gone insane, I thought this was on gentoo-user for some reason. Anyway, it's a fine suggestion =) From robert at schetterer.org Tue Sep 25 08:41:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 25 Sep 2012 07:41:26 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <50614406.50400@schetterer.org> Am 24.09.2012 21:24, schrieb Spyros Tsiolis: > > > > > ----- Original Message ----- >> From: Robert Schetterer >> To: dovecot at dovecot.org >> Cc: >> Sent: Monday, 24 September 2012, 21:06 >> Subject: Re: [Dovecot] 76Gb to 146Gb >> >> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >>> Hello all, >>> >> ----%<----%<----%<----%<----%<----%<----%<----%<---- >> >> rsync >> should do the job >> >> depending on your whole machine setup it might only be only >> umount old /home and mount new(bigger) /home after sync >> ,perhaps with tmp store elsewhere >> ( for sure you have to have a plan before doing..) >> >> but your dovecot is very outdated, i would recommend >> get up to new hard and software/os install, and then migrate >> to new machine >> >>> >> >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > On client machines I have thunderbird. > > What if : > > 1. I would make sure that thunderbird keeps a local > copy of all the message (I think there is a check box > somewhere on settings) > > 2. Make sure all client machines have synced their > mailboxes locally on thunderbird. > > > 3. Install a new version of Dovecot/Horde/XMail etc. > > 4. When the new installation is done, try to sync > from the existing clients pc's to dovecot ? > > Would that work ? > It's one scenario I am seriously contemplating. > > Thank you very much again, > > s. > in short words ,dont do it like this setup your new server, test it, then do i.e imapsync from old to new switch your ips then ,done only one way which you can go look archives and www for migration tips > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From joh.hendriks at gmail.com Tue Sep 25 10:34:02 2012 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Tue, 25 Sep 2012 09:34:02 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50615E6A.5030109@gmail.com> Spyros Tsiolis schreef: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis A solution you could try is add a new server to the mix. Leave the OS on the DL360 and put the mail data on a second machine. Then use NFS to mount the new data store. If you use a ZFS solution, expanding your data store is a no brainer. 146 GB drives are expensive, for 40 users SATA drives can do the job without any problem. gr Johan Hendriks From voytek at sbt.net.au Tue Sep 25 11:07:23 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Tue, 25 Sep 2012 18:07:23 +1000 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? Message-ID: I have a user who occasionally reads, then deletes emails using his phone, on his and others email accounts, causing emails deleted on mail server, then he says 'oops, can you recover emails I've deleted?' what's the best way to duplicate all emails to duplicate accounts ? backup folders ? so, next time he deletes emails they can be 'recovered' on the server ? I was thinking along the lines of duplicating ALL emails...? any tips and pointers appreciated -- Voytek From Ralf.Hildebrandt at charite.de Tue Sep 25 11:23:53 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 25 Sep 2012 10:23:53 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <20120925082353.GF7796@charite.de> * Voytek Eymont : > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? > > I was thinking along the lines of duplicating ALL emails...? > > any tips and pointers appreciated http://wiki.dovecot.org/Plugins/Lazyexpunge -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From h.reindl at thelounge.net Tue Sep 25 11:24:18 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 25 Sep 2012 10:24:18 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <50616A32.7040808@thelounge.net> Am 25.09.2012 10:07, schrieb Voytek Eymont: > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? configure his phone to leave messages on the server if this is the case and he is deleting them manually explain him that it is NOT your problem if he is too stupid to use email at all and sooner or later he will learn it -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From marc at perkel.com Wed Sep 26 00:34:55 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 25 Sep 2012 14:34:55 -0700 Subject: [Dovecot] Dovecot Authentication through different IMAP server? Message-ID: <5062237F.4000000@perkel.com> Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? Here's what I'm thinking. I run a spam filtering service and thinking about a way to give users access to their stored spam on my servers. However real email is forwarded to their server, So I'm wondering if I can configure dovecot somehow to do a back end authentication to their email account to gain access to their spam. Open to suggestions From noel.butler at ausics.net Wed Sep 26 04:23:11 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 26 Sep 2012 11:23:11 +1000 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1348622591.6847.23.camel@tardis> Spyros, Sounds like you have bigger problems, as you appear to have no disaster recovery processes, since your using a standalone server, as recommended to you earlier by Robert, rsync is your friend, and not just now, but daily :) setup a rolling 7 day rsync archive (sata disks are big, cheap and good enough for backups), and dont just rsync the same stuff to the same place every night, thats no good if an error occurs and wipes out your mail, coz rsync will only see it gone, and clean it out of your backup. So long as RedHat have patched dovecot for bugs and still does, 1.2.16 would still be fine, dovecot-1.2.17 is latest 1.2.x series but IIRC no longer supported, but thats fine, there are a lot, and I mean, a LOT of companies and ISP's running that version, moving to 2.x was never going to happen with new releases coming every week or so at one stage, and as 2.1.x doesn't now have nearly as regular updates, I recently myself consider 2.1 finally to be mature enough to put on production servers, which I will likely do around Christmas when its quiet. Lastly, never ever ever rely on your users to have a copy of their mail as a recovery method, it wont work, some will fsck it up, some wont bother, and you will be blamed for not having redundancy inplace. On Mon, 2012-09-24 at 20:24 +0100, Spyros Tsiolis wrote: > > > > ----- Original Message ----- > > From: Robert Schetterer > > To: dovecot at dovecot.org > > Cc: > > Sent: Monday, 24 September 2012, 21:06 > > Subject: Re: [Dovecot] 76Gb to 146Gb > > > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > >> Hello all, > >> > > ----%<----%<----%<----%<----%<----%<----%<----%<---- > > > > rsync > > should do the job > > > > depending on your whole machine setup it might only be only > > umount old /home and mount new(bigger) /home after sync > > ,perhaps with tmp store elsewhere > > ( for sure you have to have a plan before doing..) > > > > but your dovecot is very outdated, i would recommend > > get up to new hard and software/os install, and then migrate > > to new machine > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From stan at hardwarefreak.com Wed Sep 26 09:34:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 26 Sep 2012 01:34:55 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> <5060D92F.2070106@hardwarefreak.com> <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <5062A20F.8060500@hardwarefreak.com> On 9/25/2012 12:29 AM, Spyros Tsiolis wrote: > My clients don't invest in hardware. I live in Greece. Things are really > bad right now. Not that if they were better they (the client) would invest > in their infrastructure. ... > Thank you very much Stan for you kind reply. > > I reply to you in private since I was told to stop the thread > on the dovecot list because it's off-topic. I'm going to bring it back on topic. To recap the issue: 1. 73GB volume, ~82% full. Volume contains OS, spool, and mailboxes 2. Two 146GB drives on hand that are impossible to employ in either a live or offline migration due to hardware limitations--drive slots (0.82*73GB=60GB)-5GB(OS)= ~55GB/40= ~1.375GB/user mailbox (This math assumes the box has already been scoured and things like source files and other temporary stuff has already been removed, along with anything and everything not necessary to the primary functions of this box) 1.375GB/user suggests these users seem to horde old emails, or simply don't keep their mailboxes tidy. Normally I'd guess that a lot of space in a situation like this is taken up by full text search indexes, but you don't appear to have these enabled. I notice in your 'dovecot -n' output that you are already employing zlib, but to what degree is currently unknown to the list. Are you compressing maildirs with a nightly script? If not, and you're compressing on an ad hoc basis, realtime compression during write could save you quite a bit of additional space, extending the operating life of your current drives. Of course a simple request to your users to clean up their mailboxes would surely help as well, if that's a possibility--some users would flog one for making such a suggestion. If you're not fully taking advantage of compression currently, you may want to consider upgrading to Dovecot 2.x which allows transparent compression to dbox and maildir storage. You may want to migrate to mdbox message store format while still using zlib, and enable single instance attachment storage (SIS). As the name suggests, SIS saves an attachment only once, regardless how many user emails in which it exists. See: http://wiki2.dovecot.org/Plugins/Zlib I'd point you to the docs for SIS but I'm unable to find them in the Wiki. Surely someone else can provide the settings and/or update the wiki. -- Stan From robert at schetterer.org Wed Sep 26 09:39:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 08:39:27 +0200 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <5062A31F.9070705@schetterer.org> Am 25.09.2012 23:34, schrieb Marc Perkel: > Just wondering - is there any way for a dovecot server to authenticate > against a different IMAP server? > > Here's what I'm thinking. I run a spam filtering service and thinking > about a way to give users access to their stored spam on my servers. > However real email is forwarded to their server, So I'm wondering if I > can configure dovecot somehow to do a back end authentication to their > email account to gain access to their spam. > > Open to suggestions > i am not sure your idea is the best way to goal your target, running a i.e amavis filter gateway before postfix with some style of quarantaine seems to me a better idea however sasl can do auth against imap i.e ( attention examples may be outdated or different by other distro config style and/or dovecot versions) MECH=?rimap? FLAGS=?-O imap.server.name? http://blog.travisz.com/2010/12/17/debian-mail-server-with-postfix-dovecot-and-saslauthd/ #vim /etc/dovecot/dovecot.conf Look for a line that starts with ?auth default?. Once you fine that line, insert the following: auth default { mechanisms = plain login passdb pam { } userdb passwd { } socket listen { client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix there might better possibilities, wait until more response from the list -- Best Regards MfG Robert Schetterer From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 13:50:44 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 12:50:44 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder Message-ID: <5062DE04.3020601@TU-Cottbus.De> Hi Timo, hi all, we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had configured it in a test environemnt like described in "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" configuration. All actions on first level directories ".INBOX", ."Sent" or ".Archives" are working as expected. If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an error message in dovecot log: Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open expunge mailbox: Character not allowed in mailbox name: '.' What can we do or change to make it work? -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From amateo at um.es Wed Sep 26 13:55:56 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 26 Sep 2012 12:55:56 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server Message-ID: <5062DF3C.3050601@um.es> Hello, I have a infrastructure using director in front of backend servers. So I normally run doveadm commands in director servers, not the backends. The problem I have is when I try to concatenate commands like this: doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; done The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: e62e0d3834ed094e5c7900007efb8a67 66 e62e0d3834ed094e5c7900007efb8a67 71 e62e0d3834ed094e5c7900007efb8a67 74 e62e0d3834ed094e5c7900007efb8a67 75 e62e0d3834ed094e5c7900007efb8a67 77 e62e0d3834ed094e5c7900007efb8a67 78 e62e0d3834ed094e5c7900007efb8a67 so I can't postprocess correctly the output. I think it could be a bug in the communication between the doveadm client and server. PS: As the time of writing this email, I have realized that I could use fetch command directly. But, anyway, I want to report the problem/bug so it can be fixed. From maciej.uhlig at us.edu.pl Wed Sep 26 14:52:18 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 13:52:18 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line Message-ID: <5062EC72.5040505@us.edu.pl> This is a dovecot sieve log line: 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: msgid=: stored mail into mailbox 'INBOX' Would it be possible to have mail size logged here? MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From robert at schetterer.org Wed Sep 26 15:51:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:51:00 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062DE04.3020601@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> Message-ID: <5062FA34.7020502@schetterer.org> Am 26.09.2012 12:50, schrieb Andreas Kossack: > Hi Timo, hi all, > > we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had > configured it in a test environemnt like described in > "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) > > There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" > configuration. All actions on first level directories ".INBOX", ."Sent" > or ".Archives" are working as expected. > > If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an > error message in dovecot log: > Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open > expunge mailbox: Character not allowed in mailbox name: '.' > > What can we do or change to make it work? > sorry didnt tested lazy_plugin yet try http://wiki2.dovecot.org/Plugins/Listescape Examples Allow '.' characters with Maildir++ layout when virtual hierarchy separator is changed to '/': mail_plugins = $mail_plugins listescape namespace private { separator = / inbox = yes } plugin { # The default escape character is '\', but you can change it. # Note that even here the expansion of % takes place, thus you need to # use "%%" if you want to have the % sign as the escape character. #listescape_char = "\\" } or change your general namespace settings to get it work , might be better choice -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 26 15:53:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:53:28 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAC8.9000108@schetterer.org> Am 26.09.2012 13:52, schrieb Maciej Uhlig: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? > > MU > perhaps try http://wiki2.dovecot.org/Plugins/MailLog plugin { # Events to log. Also available: flag_change save mailbox_create mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Also available: flags vsize from subject mail_log_fields = uid box msgid size } -- Best Regards MfG Robert Schetterer From stephan at rename-it.nl Wed Sep 26 15:53:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 26 Sep 2012 14:53:56 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAE4.1010201@rename-it.nl> Op 9/26/2012 1:52 PM, Maciej Uhlig schreef: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? You can use the deliver_log_format setting. The default value is: "msgid=%m: %$" I'm not sure where this is documented, so I took a look at the source code to find out which substitutions exist: static struct var_expand_table static_tab[] = { { '$', NULL, NULL }, { 'm', NULL, "msgid" }, { 's', NULL, "subject" }, { 'f', NULL, "from" }, { 'e', NULL, "from_envelope" }, { 'p', NULL, "size" }, { 'w', NULL, "vsize" }, { '\0', NULL, NULL } }; So, this should work for you: deliver_log_format=msgid=%m size=%p: %$ Regards, Stephan From maciej.uhlig at us.edu.pl Wed Sep 26 16:17:36 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 15:17:36 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062FAE4.1010201@rename-it.nl> References: <5062EC72.5040505@us.edu.pl> <5062FAE4.1010201@rename-it.nl> Message-ID: <50630070.4040000@us.edu.pl> Stephan Bosch: > > So, this should work for you: > > deliver_log_format=msgid=%m size=%p: %$ > Oh yes it did the trick :-) Thank you and best regards, MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 17:15:27 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 16:15:27 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062FA34.7020502@schetterer.org> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> Message-ID: <50630DFF.2050405@TU-Cottbus.De> Hi Robert, >> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >> error message in dovecot log: >> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >> expunge mailbox: Character not allowed in mailbox name: '.' > sorry didnt tested lazy_plugin yet > try > http://wiki2.dovecot.org/Plugins/Listescape It works now with "mail_plugins = $mail_plugins ... lazy_expunge listescape" Thank you for your hint in the right direction. -- Mit freundlichen Gr??en A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ Diese Nachricht wurde aus 100% wiederverwerteten Elektronen erstellt. _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From CMarcus at Media-Brokers.com Wed Sep 26 17:21:07 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 26 Sep 2012 10:21:07 -0400 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50616A32.7040808@thelounge.net> References: <50616A32.7040808@thelounge.net> Message-ID: <50630F53.8060906@Media-Brokers.com> On 2012-09-25 4:24 AM, Reindl Harald wrote: > Am 25.09.2012 10:07, schrieb Voytek Eymont: >> I have a user who occasionally reads, then deletes emails using his phone, >> on his and others email accounts, causing emails deleted on mail server, >> then he says 'oops, can you recover emails I've deleted?' >> >> what's the best way to duplicate all emails to duplicate accounts ? backup >> folders ? so, next time he deletes emails they can be 'recovered' on the >> server ? > configure his phone to leave messages on the server > > if this is the case and he is deleting them manually explain > him that it is NOT your problem if he is too stupid to use > email at all and sooner or later he will learn it Not an option if said user is the boss... -- Best regards, Charles From robert at schetterer.org Wed Sep 26 17:31:41 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 16:31:41 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <506311CD.9060608@schetterer.org> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> Am 25.09.2012 10:07, schrieb Voytek Eymont: >>> I have a user who occasionally reads, then deletes emails using his >>> phone, >>> on his and others email accounts, causing emails deleted on mail server, >>> then he says 'oops, can you recover emails I've deleted?' >>> >>> what's the best way to duplicate all emails to duplicate accounts ? >>> backup >>> folders ? so, next time he deletes emails they can be 'recovered' on the >>> server ? >> configure his phone to leave messages on the server >> >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... > you may do it like this, other soltuions may also thinkable ( depends on your local needs and setup) configure email archive i.e with postfix with i.e always_bcc = address Deliver a copy of all mail to the specified address. In Postfix versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), or pickup(8). sender_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope sender address for an automatic BCC address. This feature is available in Postfix 2.1 and later. recipient_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope recipient address for an automatic BCC address. This feature is available in Postfix 2.1 and later. deliver bc whatever to a second imap account mailbox in a folder set read only via acl i.e with sieve, if you want ,sort by date too etc so if mail was lost ,why ever, the user is able to login into his archive and copy that mail back to his real account -- Best Regards MfG Robert Schetterer From kgc at corp.sonic.net Wed Sep 26 20:34:18 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 10:34:18 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 Message-ID: <50633C9A.5060700@corp.sonic.net> Timo - I upgraded to 2.1.10 on our director servers two nights ago and apart from errors associated with the directors processes restarting everything looked great for ~24 hours until I failed our the real servers last night to update the nfs mount options for the spools. I followed the suggested procedure for each backend server, just run on one of the directors, which seemed to work as expected. doveadm director add x.x.x.x 0 doveadm director flush x.x.x.x The following errors on the directors that started after this went unnoticed until this AM. director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) ... User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. -K From tss at iki.fi Wed Sep 26 20:57:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 20:57:58 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <50633C9A.5060700@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> Message-ID: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> On 26.9.2012, at 20.34, Kelsey Cummings wrote: > The following errors on the directors that started after this went unnoticed until this AM. > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > ... > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? From kgc at corp.sonic.net Wed Sep 26 21:06:33 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 11:06:33 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> Message-ID: <20120926180633.GE80443@corp.sonic.net> On Wed, Sep 26, 2012 at 08:57:58PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 20.34, Kelsey Cummings wrote: > > > The following errors on the directors that started after this went unnoticed until this AM. > > > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > > ... > > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. > > After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? No, there continued to be a mix of both. The pattern seems to look like this. I'll run some stats later but it looks like a pretty significant number of users where affected. 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Wed Sep 26 21:13:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 21:13:26 +0300 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <50630DFF.2050405@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> Message-ID: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> On 26.9.2012, at 17.15, Andreas Kossack wrote: >>> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >>> error message in dovecot log: >>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>> expunge mailbox: Character not allowed in mailbox name: '.' > >> sorry didnt tested lazy_plugin yet >> try >> http://wiki2.dovecot.org/Plugins/Listescape > It works now with > "mail_plugins = $mail_plugins ... lazy_expunge listescape" Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd From h.reindl at thelounge.net Wed Sep 26 21:15:24 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 26 Sep 2012 20:15:24 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <5063463C.9090002@thelounge.net> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... surely even the boss has to understand that there is no technical solution for a social problem - if not: wrong company -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From stsiol at yahoo.co.uk Thu Sep 27 00:19:06 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Wed, 26 Sep 2012 22:19:06 +0100 (BST) Subject: [Dovecot] second dovecot imap server Message-ID: <1348694346.16608.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello again all, I would like to thank everyone for rushing to help me with the dovecot migration issue. I am , currently, reading and studying what has been said by all the good and kind people who responded to? this list. OK, on the same issue,I thought of adding a totally new server with dovecot acting as a secondary/backup server for the mail servicing. Would that be possible ?? What if I manage to make it work; Can I retire the old(er) server ? I would think that this is more on-topic than my last post. If I am out of line again, please respond in private. Thank you all! spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From kgc at corp.sonic.net Thu Sep 27 00:38:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 14:38:37 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <506375DD.3080804@corp.sonic.net> On 09/26/12 11:06, Kelsey Cummings wrote: > No, there continued to be a mix of both. The pattern seems to look like > this. I'll run some stats later but it looks like a pretty significant > number of users where affected. Timo, it looks like the total number of affected users was only about 250 and that most of their erred connections were surrounded by successful sessions. -K From eliezer at ngtech.co.il Thu Sep 27 01:25:07 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 27 Sep 2012 00:25:07 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506380C3.8010903@ngtech.co.il> On 9/24/2012 7:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot It really depends on the raid you have. is it software or hardware raid? if it's software raid it will be more then simple to do it while taking one drive out put a new one in and use let say FINNIX linux from cd\dvd\usb and manage the whole partitioning copying etc from another OS while not harming anything in the old HDDs. you will might need to setup a new partitions manually on the new drive but just make a plan try it on some small VM to make sure the steps you are doing are fine for centos 5.5 and go for it. rsync is your friend!! in this case. someone mentioned here Gentoo which one installation of the system can give you very big backgorund on manual partitioning chrooting and other basic stuff that can help you in the process. Regards, Eliezer > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From amateo at um.es Thu Sep 27 08:52:59 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 27 Sep 2012 07:52:59 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: <5063E9BB.8010104@um.es> I forgot to mention that I'm running dovecot 2.1.9 in my servers. El 26/09/12 12:55, Angel L. Mateo escribi?: > Hello, > > I have a infrastructure using director in front of backend servers. > So I normally run doveadm commands in director servers, not the > backends. The problem I have is when I try to concatenate commands like > this: > > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w > | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} > size.physical mailbox-guid $guid uid $uid; > done > > The problem is that although when I run doveadm search command in > the backend server I correctly get the list of mails, each line with the > mailbox-guid and the uid of the message, when I run the same command in > the director server, format of the list is corrupted and there are lines > that contains just the mailbox-guid and the next the uid (of the > previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 > > so I can't postprocess correctly the output. I think it could be a > bug in the communication between the doveadm client and server. > > PS: As the time of writing this email, I have realized that I could use > fetch command directly. But, anyway, I want to report the problem/bug so > it can be fixed. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From Jost.Krieger+dovecot at rub.de Thu Sep 27 12:08:43 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 11:08:43 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125241.GF2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <20120919125241.GF2010@ruhr-uni-bochum.de> Message-ID: <20120927090843.GW12477@ruhr-uni-bochum.de> On Wed 19 Sep 2012 02:52:41 PM GMT, Jost Krieger wrote: > Oops, that's not what I want! > > How about another command line parameter for lda to suppress the > emergency save? We know exactly when we are trying to save probable > spam. I've tried my luck. Would the attached patch be sensible and welcome? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | -------------- next part -------------- diff -r 88a05f387743 doc/man/dovecot-lda.1.in --- a/doc/man/dovecot-lda.1.in Thu Sep 27 02:55:14 2012 +0300 +++ b/doc/man/dovecot-lda.1.in Thu Sep 27 11:04:56 2012 +0200 @@ -5,7 +5,7 @@ .\"------------------------------------------------------------------------ .SH SYNOPSIS .B dovecot\-lda -.RB [ \-ek ] +.RB [ \-eEk ] [\fB\-a\fP \fIaddress\fP] [\fB\-c\fP \fIconfig_file\fP] [\fB\-d\fP \fIusername\fP] @@ -61,6 +61,10 @@ The default is to send a rejection mail ourself. .\"------------------------------------- .TP +.B \-E +If mail delivery fails, don't try to do an emergency delivery attempt to INBOX. +.\"------------------------------------- +.TP .BI \-f\ envelope_sender Envelope sender address. .\"------------------------------------- diff -r 88a05f387743 src/lda/main.c --- a/src/lda/main.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lda/main.c Thu Sep 27 11:04:56 2012 +0200 @@ -310,7 +310,7 @@ master_service = master_service_init("lda", MASTER_SERVICE_FLAG_STANDALONE | MASTER_SERVICE_FLAG_DONT_LOG_TO_STDERR, - &argc, &argv, "a:d:ef:km:p:r:"); + &argc, &argv, "a:d:eEf:km:p:r:"); memset(&ctx, 0, sizeof(ctx)); ctx.session = mail_deliver_session_init(); @@ -334,6 +334,9 @@ case 'e': stderr_rejection = TRUE; break; + case 'E': + ctx.suppress_emergency_delivery = TRUE; + break; case 'f': /* envelope sender address */ ctx.src_envelope_sender = diff -r 88a05f387743 src/lib-lda/mail-deliver.c --- a/src/lib-lda/mail-deliver.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.c Thu Sep 27 11:04:56 2012 +0200 @@ -402,7 +402,7 @@ ret = mail_deliver_save(ctx, ctx->dest_mailbox_name, 0, NULL, storage_r); } - if (ret < 0 && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { + if (ret < 0 && !ctx->suppress_emergency_delivery && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { /* still didn't work. try once more to save it to INBOX. */ ret = mail_deliver_save(ctx, "INBOX", 0, NULL, storage_r); diff -r 88a05f387743 src/lib-lda/mail-deliver.h --- a/src/lib-lda/mail-deliver.h Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.h Thu Sep 27 11:04:56 2012 +0200 @@ -49,6 +49,7 @@ struct var_expand_table *var_expand_table; bool tried_default_save; + bool suppress_emergency_delivery; bool saved_mail; bool save_dest_mail; /* Delivery failed because user is out of quota / disk space */ From davide.marchi at mail.cgilfe.it Thu Sep 27 12:22:47 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Thu, 27 Sep 2012 11:22:47 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) Message-ID: <50641AE7.6040201@mail.cgilfe.it> Hi to all, sorry in advance for my poor english, this is the first time that i wrote to a list if i make mistake .... excuseme. My problem is this: i have dovecot 2.1.8 installed and functioning from 2 years one week ago i have installed crm114 for my last spam detection filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" My mail system is qmail that through .qmail default correctly tag with X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 X-CRM114-Status: GOOD ( 28.64 ) headers dovecot.conf is as follow: plugin { . . . antispam_signature = X-CRM114-CacheID antispam_verbose_debug = 1 antispam_debug_target = stderr antispam_backend = crm114 antispam_signature_missing = error antispam_trash = Trash antispam_spam = SPAM antispam_unsure = UNSURE antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = --fileprefix=/opt/crm114/ . . . } protocol imap { . . . mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = $mail_plugins quota imap_quota mail_log antispam . . . } I have a sieve default compiled script that automatically put mails in SPAM folder or UNSURE folder if respectively in message hedere there is the evaluation of crm114 (UNSURE or SPAM). When i move mails incorrectly tagged by crm (for example from SPAM to my inbox or from UNSURE to SPAM) thunderbird (company official mail client ver 15.01) go in error with the following message: "Operation failed over folder 'UNSURE'. Server for account davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 binary.." I dont'know where is my mistake if someone can help me is much appreciated Thank you -- */Davide Marchi /Teorema Ferrara Srl /(Tel: /**/+39 0532 783161)/**/ (Fax: +/**/39 0532 783368/**/)/**//**/ /**//**/Davide.Marchi at mail.cgilfe.it /CONFIDENZIALITA? *** Questo messaggio, inclusi gli eventuali allegati, ? indirizzato solo ai destinatari e pu? contenere informazioni riservate e confidenziali. Se avete ricevuto il messaggio senza esserne un destinatario, siete pregati di non utilizzare, copiare o distribuire questo messaggio o i suoi allegati. Se avete ricevuto il messaggio per errore, siete pregati di cancellarlo dal vostro sistema, assieme a tutti gli allegati, e di informare immediatamente il mittente,sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. *CONFIDENTIALITY*** This message and any attachments are intended only for the use of the addressee(s) and may contain information that is privileged and/or confidential. If the reader of the message is not the intended recipient(s) or an authorized representative of the intended recipient(s), please do not use, copy, distribute this email or its attachments or take action based on them. If you have received this communication in error, please notify us immediately by email and delete the message and any attachments from your system. *Teorema FerraraSrl - Via Spronello, 7 - 44121 Ferrara Italia - P. IVA 01244490387** P**S**tampa questo messaggio solo se veramente necessario, pensa alla natura! - Please consider whether it is necessary to print this e-mail* From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 14:41:15 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 13:41:15 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox Message-ID: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Hello. After we switched the format from mbox to mdbox on our dovecot (version 2.1.9) servers last week, some of our users were missing the subfolders of their inboxes. So far we thought that with the mbox format it is not possible to have mail folders containing both messages and subfolders. Now we know that there is one exception from this rule: the inbox. With mbox format dovecot supports subfolders of the inbox, and a few of our users actually detected and used this feature! Internally the messages are stored in /var/mail/user or in a file named 'inbox' and the subfolders are in the directory 'INBOX'. (The OS is case sensitiv.) The problem was that doveadm and dsync do not support subfolders of the inbox and therefore these folders did not get synchronized from our old mbox server to the new mdbox dovecot server. The command doveadm -o mail_location=mbox:~user123/Mail mailbox list -u user123 '*' did not show any subfolder of INBOX, whereas the the following with a wrong mail_location showed that there are subfolders of INBOX doveadm -o mail_location=mbox:~user123 mailbox list -u user123 '*' Finally we could import the missing subfolders one after one in the mdbox server with commands like this: doveadm import -u user123 mbox:~user123/Mail "" mailbox INBOX/subfolder all It would be nice if doveadm could be changed to not overlook subfolders of the inbox with mbox format. Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From Jost.Krieger+dovecot at rub.de Thu Sep 27 15:13:13 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 14:13:13 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Message-ID: <20120927121131.GC12477@ruhr-uni-bochum.de> On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > After we switched the format from mbox to mdbox on our dovecot > (version 2.1.9) servers last week, some of our users were missing > the subfolders of their inboxes. So far we thought that with the > mbox format it is not possible to have mail folders containing both > messages and subfolders. Now we know that there is one exception > from this rule: the inbox. With mbox format dovecot supports > subfolders of the inbox, and a few of our users actually detected > and used this feature! Please see the thread "No status for INBOX/* in mbox NS (2.1.6)" there's a patch from Timo. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 15:56:07 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 14:56:07 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927121131.GC12477@ruhr-uni-bochum.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> <20120927121131.GC12477@ruhr-uni-bochum.de> Message-ID: <20120927145607.89971mht64oiixnk@webmail.hrz.uni-giessen.de> Am Do, 27 Sep 2012 schrieb Jost Krieger: > On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > >> After we switched the format from mbox to mdbox on our dovecot >> (version 2.1.9) servers last week, some of our users were missing >> the subfolders of their inboxes. So far we thought that with the >> mbox format it is not possible to have mail folders containing both >> messages and subfolders. Now we know that there is one exception >> from this rule: the inbox. With mbox format dovecot supports >> subfolders of the inbox, and a few of our users actually detected >> and used this feature! > > Please see the thread > "No status for INBOX/* in mbox NS (2.1.6)" > there's a patch from Timo. Thanks for the info. This patch seems to be in dovecot 2.1.10, we made our format switch from mbox to mdbox with version 2.1.9. Obviously one week too early ;-). Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From blevi.linux at gmail.com Thu Sep 27 16:25:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Thu, 27 Sep 2012 16:25:34 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <506453CE.7000608@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > Hi Timo I just want to ask you: this issue is still in your task list? If you need more debug information please tell me how can I give you. Thanks, Levi From humberto.pedroso at gmail.com Thu Sep 27 17:36:09 2012 From: humberto.pedroso at gmail.com (Humberto A. Pedroso) Date: Thu, 27 Sep 2012 11:36:09 -0300 Subject: [Dovecot] Deleted to trash plugin Message-ID: Hello, Does anybody are using deleted_to_trash plugin with Dovecot 2.1.9? I'm trying to compile and I'm with some difficulties. I tested with dovecot 2.0 and it works very well. -- Humberto From bob at computerisms.ca Thu Sep 27 19:37:20 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 09:37:20 -0700 Subject: [Dovecot] sieve vacation Message-ID: <1348763840.533.187.camel@worklian> Hello, I set up pigeonhole on a server and am using roundcube's sieverules to create a set of rules. When using the vacation feature, I can see in the logs that dovecot is sending the vacation response, but the recipient never receives it. What method does dovecot use to send the response? Does it use the sendmail binary, does it use the mail command, does it connect to some smtp server somewhere, does it use qmail-inject if it finds it? I find no evidence it is using any of these methods on my server, but I am not sure if one of these methods is failing or if there is some place else I should be looking for the missing vacation response... Thank you for any suggestions... -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From robert at schetterer.org Thu Sep 27 19:52:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 18:52:29 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348763840.533.187.camel@worklian> References: <1348763840.533.187.camel@worklian> Message-ID: <5064844D.2000502@schetterer.org> Am 27.09.2012 18:37, schrieb Bob Miller: > the logs that dovecot is sending the vacation response, but the > recipient never receives it. show the logs study http://wiki2.dovecot.org/LDA Common configuration The settings are listed in the example conf.d/15-lda.conf file. The important settings are: postmaster_address is used as the From: header address in bounce mails hostname is used in generated Message-IDs and in Reporting-UA: header in bounce mails sendmail_path is used to send mails. Note that the default is /usr/sbin/sendmail, which doesn't necessarily work the same as /usr/lib/sendmail. Alternatively you can use submission_host to send mails via the specified SMTP server. auth_socket_path specifies the UNIX socket to auth-userdb where LDA can lookup userdb information when -d parameter is used. See below how to configure Dovecot to configure the socket. -- Best Regards MfG Robert Schetterer From bob at computerisms.ca Thu Sep 27 20:12:51 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 10:12:51 -0700 Subject: [Dovecot] sieve vacation In-Reply-To: <5064844D.2000502@schetterer.org> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> Message-ID: <1348765971.533.189.camel@worklian> Robert, > The settings are listed in the example conf.d/15-lda.conf file. The > important settings are: > > sendmail_path is used to send mails. Note that the default is > /usr/sbin/sendmail, which doesn't necessarily work the same as > /usr/lib/sendmail. > > Alternatively you can use submission_host to send mails via the > specified SMTP server. This is the missing piece of information I wasn't finding. Thank you very much... From robert at schetterer.org Thu Sep 27 21:11:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 20:11:00 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348765971.533.189.camel@worklian> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> <1348765971.533.189.camel@worklian> Message-ID: <506496B4.4000508@schetterer.org> Am 27.09.2012 19:12, schrieb Bob Miller: > Robert, > > >> The settings are listed in the example conf.d/15-lda.conf file. The >> important settings are: >> >> sendmail_path is used to send mails. Note that the default is >> /usr/sbin/sendmail, which doesn't necessarily work the same as >> /usr/lib/sendmail. >> >> Alternatively you can use submission_host to send mails via the >> specified SMTP server. > > This is the missing piece of information I wasn't finding. Thank you > very much... > youre welcome Bob *g -- Best Regards MfG Robert Schetterer From jbates at brightok.net Thu Sep 27 23:30:18 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 27 Sep 2012 15:30:18 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused Message-ID: <5064B75A.7060307@brightok.net> Version: 2.1.9 Connecting directly to the proxy server on the LMTP port, issue lhlo, mail from, rcpt to, data. After the . closing of data, it just hangs there. The logs on the proxy server show: Sep 27 19:55:12 proxy1 dovecot: lmtp(9398): Connect from 69.8.2.71 Sep 27 19:55:24 proxy1 dovecot: lmtp(9398): Error: lmtp client: connect(lmtp, 7025) failed: Connection refused root 9398 1102 0 19:53 ? 00:00:00 dovecot/lmtp The proxy server is hard coded (not director in this case). I purposefully shut it down to see how the system would handle failures. Unfortunately, it appears that the process just hangs there. For fun, I tried a separate protocol lmtp option in the config to try proxy_timeout. It made no difference. Missing some error handling? protocol lmtp { passdb { driver = static args = proxy=y host=%s nopassword=y proxy_timeout=120 } } Jack From Andreas.Kossack at TU-Cottbus.De Fri Sep 28 10:25:55 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Fri, 28 Sep 2012 09:25:55 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> Message-ID: <50655103.5090205@TU-Cottbus.De> Hi Timo, hi all, >>>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>>> expunge mailbox: Character not allowed in mailbox name: '.' >> >>> try >>> http://wiki2.dovecot.org/Plugins/Listescape >> It works now with >> "mail_plugins = $mail_plugins ... lazy_expunge listescape" > > Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd Now it works lika a charm without "listescape" "mail_plugins = $mail_plugins ... lazy_expunge" Thank you Timo... -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ From jbates at brightok.net Fri Sep 28 19:37:50 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 11:37:50 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5064B75A.7060307@brightok.net> References: <5064B75A.7060307@brightok.net> Message-ID: <5065D25E.1030507@brightok.net> On 9/27/2012 3:30 PM, Jack Bates wrote: > Version: 2.1.9 > > Connecting directly to the proxy server on the LMTP port, issue lhlo, > mail from, rcpt to, data. After the . closing of data, it just hangs > there. The logs on the proxy server show: > Looking at dovecot's proxy code. Calls to connect_connect_ip do not return an error when the server isn't there. This may be expected, but it is what determines failure/OK in the LMTP code. The log entry is performed by lmtp_client_wait_connect callback. In the standard client-common-auth.c code, the error returned for a failure is 1) syslog error is reported by proxy_wait_connect() callback 2) error to client is returned by an error in proxy_input() In the lmtp code, we issue a "250 2.1.5 OK". Right after, the lmtp_client_wait_connect() callback is called and issues the connection refused error. However, we've already approved the rcpt. I am not sure which method we really want in the lmtp proxy. Should it be failing at the initial rcpt command or after the data command. Currently, though I haven't checked yet, we don't appear to be detecting the input failure and handling the data command correctly since we've already allowed the rcpt to proceed without a backend proxy available. Jack From jbates at brightok.net Fri Sep 28 20:44:21 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 12:44:21 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065D25E.1030507@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> Message-ID: <5065E1F5.4010506@brightok.net> On 9/28/2012 11:37 AM, Jack Bates wrote: > On 9/27/2012 3:30 PM, Jack Bates wrote: >> Version: 2.1.9 >> >> Connecting directly to the proxy server on the LMTP port, issue lhlo, >> mail from, rcpt to, data. After the . closing of data, it just hangs >> there. The logs on the proxy server show: > I am not sure which method we really want in the lmtp proxy. Should it > be failing at the initial rcpt command or after the data command. > Currently, though I haven't checked yet, we don't appear to be > detecting the input failure and handling the data command correctly > since we've already allowed the rcpt to proceed without a backend > proxy available. Further testing shows that the connection fails are normally handled after the DATA block. It works perfectly if you only fail one recipient. If all recipients are failed, the code locks up instead of telling the client that they are all failed. Now to find out where between lmtp_client_fail() and some other part of the code, we aren't treating it right. :( Jack From tss at iki.fi Fri Sep 28 21:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 21:59:52 +0300 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 In-Reply-To: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> References: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Message-ID: <43585454-3155-4B89-B5FD-2D2ACA80E562@iki.fi> On 22.9.2012, at 19.11, Clemens Schrimpe wrote: > Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: > > sudo /usr/libexec/StartupItemContext dovecot > > (again: all this was working fine under 2.0.15) > > Now with 2.1.10, when I manually invoke dovecot with just > > sudo dovecot > > Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. I don't really know about mach contexts or how they're supposd to work.. > What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? Not much I think. I just looked at the diff between 2.0 and 2.1 and don't really see anything I could blame. If you have time you could try bisecting with mercurial (basically try different versions from hg) to isolate the change that broke it. I don't really have time to debug this.. From tss at iki.fi Fri Sep 28 22:01:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:01:13 +0300 Subject: [Dovecot] Dovecot Clustering with dsync over ssh In-Reply-To: References: Message-ID: On 23.9.2012, at 10.56, weber at zackbummfertig.de wrote: > i only found an old post of tiemo where he announced that he will do an wiki article > how to setup dovecot cluster with dsync over ssh. > anyone knows if this artcile exists and where? > > i would like to setup a dovecot cluster with 2 machines. There are some mailing list posts about how to do it. Anyway I don't advertise it much yet because v2.1's dsync isn't really quite good enough to do it well. v2.2 dsync should work well. From tss at iki.fi Fri Sep 28 22:02:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:02:42 +0300 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <9952E585-05B8-4418-A3BD-0ED0F0A84165@iki.fi> On 26.9.2012, at 0.34, Marc Perkel wrote: > Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? v2.1 has passdb imap From jbates at brightok.net Fri Sep 28 23:12:44 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:12:44 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065E1F5.4010506@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> Message-ID: <506604BC.6050503@brightok.net> On 9/28/2012 12:44 PM, Jack Bates wrote: > On 9/28/2012 11:37 AM, Jack Bates wrote: >> On 9/27/2012 3:30 PM, Jack Bates wrote: >>> Version: 2.1.9 >>> >>> Connecting directly to the proxy server on the LMTP port, issue >>> lhlo, mail from, rcpt to, data. After the . closing of data, it just >>> hangs there. The logs on the proxy server show: >> > Further testing shows that the connection fails are normally handled > after the DATA block. It works perfectly if you only fail one > recipient. If all recipients are failed, the code locks up instead of > telling the client that they are all failed. > > Now to find out where between lmtp_client_fail() and some other part > of the code, we aren't treating it right. :( It appears that the callbacks for the proxy connections are handled in the ioloop stuff. If you have at least one valid proxy session open, then proxy_try_finish will get called and all the replies are sent. The problem appears to be if all backend proxy sessions are bad (ie single recipient, proxy server down), there is nothing for ioloop to callback to. proxy_try_finish will never get called, and we end up locking up at epoll_wait as there are no events for us to process. Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. Jack From jbates at brightok.net Fri Sep 28 23:29:11 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:29:11 -0500 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <506604BC.6050503@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> Message-ID: <50660897.6040008@brightok.net> On 9/28/2012 3:12 PM, Jack Bates wrote: > > Code needs to be written to handle the special case of us not having > any proxy callbacks as they are all bad. > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 *************** *** 300,303 **** --- 300,304 ---- lmtp_client_send(conn->client, conn->data_input); lmtp_client_send_more(conn->client); } + lmtp_proxy_try_finish(proxy); } ie, call lmtp_proxy_try_finish once. If all is bad, this will wrap us up. If we have valid proxies, it'll probably not finish and we'll return to waiting on callbacks. I'm not sure of any blocking restrictions. I just know it works. mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) 250 2.0.0 CYKfHcsHZlBcCAAALhEySA Saved mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 250 2.0.0 EYKfHcsHZlBcCAAALhEySA Saved Jack From CMarcus at Media-Brokers.com Sun Sep 30 16:02:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 30 Sep 2012 09:02:03 -0400 Subject: [Dovecot] Log NAT IP address? Message-ID: <506842CB.8080501@Media-Brokers.com> Hi Timo/everyone, Currently we are logging the remote IP, but is there a way to show the IP address that the NAT connection is coming from? The reason I ask is, we are changing ISPs, and I would like to see in the logs when an external connection is coming from our OLD ISP connection, and when it is coming through our new one. We have a Watchguard firewall, and I have both External connections setup and working, and have just pointed our DNS records to the new public IP, and would like to be able to see which WAN connection/IP they are coming from. Thanks, -- Best regards, Charles From jbates at brightok.net Sun Sep 30 17:33:14 2012 From: jbates at brightok.net (Jack Bates) Date: Sun, 30 Sep 2012 09:33:14 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068582A.6030507@brightok.net> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. > > We have a Watchguard firewall, and I have both External connections > setup and working, and have just pointed our DNS records to the new > public IP, and would like to be able to see which WAN connection/IP > they are coming from. You could bind 2 internal IP Addresses to the server and have each NAT translation go to a different internal IP. Jack From dovec at nettrust.co.nz Sat Sep 1 01:42:32 2012 From: dovec at nettrust.co.nz (Michael) Date: Sat, 01 Sep 2012 10:42:32 +1200 Subject: [Dovecot] dovecotadm error Message-ID: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> When I use the following command: doveadm expunge -A mailbox Junk savedbefore 28d I get: doveadm(root): Error: User listing returned failure In the mail log file it says: dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table 'system.users' doesn't exist (using built-in default iterate_query: SELECT username, domain FROM users) However this is setup: dovecot-sql.conf: iterate_query = SELECT username AS user FROM accounts though I have had to comment the following from dovecot.conf: #iteratedb { # args = /usr/local/etc/dovecot/dovecot-sql.conf # driver = sql #} because it errors on dovecot start up. I have already spent a bit of time on the Dovecot website and Google trying to find an answer. I want to get the expires plugin working. Any leads please? dovecot -n # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.29.6-rt24-smp i686 Slackware 13.1.0 auth_master_user_separator = * auth_mechanisms = plain login dict { expire = mysql:/usr/local/etc/dovecot/dovecot-dict-expire.conf quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 mail_access_groups = dovecot mail_location = Maildir:%h/Maildir mail_plugins = " expire" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Drafts autocreate2 = Junk autocreate3 = Sent autocreate4 = Trash autosubscribe = Drafts autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Trash expire = Junk expire_dict = proxy::expire quota = dict:user::proxy::quotadict sieve = %h/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/before.sieve sieve_dir = %h/sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service dict { unix_listener dict { group = dovecot mode = 0660 } } service imap-login { inet_listener imap { address = DELETED } } service managesieve-login { executable = /usr/local/libexec/dovecot/managesieve-login } service managesieve { executable = /usr/local/libexec/dovecot/managesieve } ssl_cert = I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) disconnect in IDLE and inactivity time outs. I'm using only IMAP with two users. Both users access the server from multiple devices, phone, tablet and desktop computer. I had previously set the max_userip_connections to 20 and rarely got the following limits exceeded message. That's probably being triggered when we are both home and using all of our devices on the same network. I've now changed it to 40 and we'll see how that does. The other is the IDLE and inactivity timeouts. I don't really understand how IDLE works and I couldn't find an inactivity timeout setting to play with. There are at this second 32 imap or imap-login processes running with only two phones and one desktop running. That's a few more than before the upgrade but not a lot. --------------------- Dovecot Begin ------------------------ Dovecot limits exceeded: max_userip_connections: penny from 216.240.58.138 to 216.240.58.140: 56 Time(s) Dovecot disconnects: Disconnected in IDLE: 485 Time(s) Inactivity: 343 Time(s) no auth attempts: 14 Time(s) no reason: 21 Time(s) Everything seems to be working OK so maybe I should just not worry about it? Thanks, knute... # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_anonymous_username = anonymous auth_cache_negative_ttl = 1 hours auth_cache_size = 0 auth_cache_ttl = 1 hours auth_debug = no auth_debug_passwords = no auth_default_realm = auth_failure_delay = 2 secs auth_first_valid_uid = 500 auth_gssapi_hostname = auth_krb5_keytab = auth_last_valid_uid = 0 auth_master_user_separator = auth_mechanisms = plain login auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 60 default_internal_user = dovecot default_login_user = dovenull default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins disable_plaintext_auth = yes dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_proxy_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 first_valid_gid = 1 first_valid_uid = 500 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = imap_idle_notify_interval = 2 mins imap_logout_format = bytes=%i/%o imap_max_line_length = 64 k import_environment = TZ info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/lib/dovecot listen = *, :: lmtp_proxy = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = syslog log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = Dovecot ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib/dovecot/modules mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no passdb { args = deny = no driver = pam master = no pass = no } pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = " imap" quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs inet_listener { address = port = 0 ssl = no } privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener director-userdb { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 0 ssl = no } inet_listener pop3s { address = port = 0 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = Hi, I've not found the answer to this question anywhere - please forgive me if I overlooked. I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net From acrow at integrafin.co.uk Sat Sep 1 12:42:16 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Sat, 01 Sep 2012 10:42:16 +0100 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: <5041D878.3090609@integrafin.co.uk> On 01/09/12 09:10, David Anderson wrote: > Hi, > > I've not found the answer to this question anywhere - please forgive > me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are > hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same > way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and > logwatch can be noisy and it's easy to overlook. > > My motivation: for some reason my POP3 listener was hanging, and I had > to restart dovecot (this is 2.0.9). But I was not aware - the process > was still there, it was accepting connections (but not doing anything > useful with them) and was actually managing to log the fact that the > process limit was being hit (as more and more connections came in). If > I could have been alerted I could have restarted dovecot earlier. > > Many thanks, > David > I believe that nagios or icinga could do this for you with a log analyser plugin. http://exchange.nagios.org/directory/Plugins/Log-Files Cheers Alex From leamhall at gmail.com Sat Sep 1 12:53:15 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 05:53:15 -0400 Subject: [Dovecot] Testing process? Message-ID: Morning all! Total newbie question. I'm learning Dovecot and will be plugging in a few things and testing it end to end. What test structures or tools are available for making sure the install is correct and the parameters are taken? Thanks! Leam -- Mind on a Mission From robert at schetterer.org Sat Sep 1 13:30:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 01 Sep 2012 12:30:18 +0200 Subject: [Dovecot] Testing process? In-Reply-To: References: Message-ID: <5041E3BA.3090100@schetterer.org> Am 01.09.2012 11:53, schrieb leam hall: > Morning all! Total newbie question. I'm learning Dovecot and will be > plugging in a few things and testing it end to end. What test structures or > tools are available for making sure the install is correct and the > parameters are taken? > > Thanks! > > Leam > > describe what you mean "install is correct" as there are unnumbered ways to setup dovecot usally you found your errors in the logs, for testing use verbose logging so first plan "what setup" you want to goal, reading dovecot sites , faqs, examples, archives anyway perhaps use some virt. machine for testing multiple stuff -- Best Regards MfG Robert Schetterer From kprprl at gmail.com Sat Sep 1 15:25:19 2012 From: kprprl at gmail.com (PARTH MONGA) Date: Sat, 1 Sep 2012 17:55:19 +0530 Subject: [Dovecot] Testing process? In-Reply-To: <5041E3BA.3090100@schetterer.org> References: <5041E3BA.3090100@schetterer.org> Message-ID: netstat -tulpn port listening on 110(pop) 143(imap) confirms the same and you can also do cat /var/log/dovecot.log rgds p On Sat, Sep 1, 2012 at 4:00 PM, Robert Schetterer wrote: > Am 01.09.2012 11:53, schrieb leam hall: > > Morning all! Total newbie question. I'm learning Dovecot and will be > > plugging in a few things and testing it end to end. What test structures > or > > tools are available for making sure the install is correct and the > > parameters are taken? > > > > Thanks! > > > > Leam > > > > > > describe what you mean "install is correct" > as there are unnumbered ways to setup dovecot > > usally you found your errors in the logs, for testing use verbose logging > > so first plan "what setup" you want to goal, reading dovecot sites , > faqs, examples, archives > > anyway perhaps use some virt. machine for testing multiple stuff > -- > Best Regards > MfG Robert Schetterer > From leamhall at gmail.com Sat Sep 1 17:19:07 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 10:19:07 -0400 Subject: [Dovecot] Testing process? In-Reply-To: References: <5041E3BA.3090100@schetterer.org> Message-ID: > > describe what you mean "install is correct" > > as there are unnumbered ways to setup dovecot > > > > usally you found your errors in the logs, for testing use verbose logging > > > > so first plan "what setup" you want to goal, reading dovecot sites , > > faqs, examples, archives > > > > anyway perhaps use some virt. machine for testing multiple stuff > > -- > > Best Regards > > MfG Robert Schetterer > > > Robert, et al, thank you for reminding me that I need to share what I'm thinking! I've been bouncing it around in my own head but didn't really write enough to communicate. There are a lot of things i need to learn about Dovecot, so first is how to send something to it and then see where it's logged or what actions are taken. My work uses Dovecot and I need to be able to use, test, and understand it. So maybe the first question is, is there a test framework for Dovecot itself or is it better to set up with postfix or something and just route mail through it? Thanks! Leam -- Mind on a Mission From david at wordshell.net Sat Sep 1 21:08:53 2012 From: david at wordshell.net (David Anderson) Date: Sat, 01 Sep 2012 21:08:53 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <1346490929.10359.16.camel@tardis> References: <5041C2FD.1040108@wordshell.net> <1346490929.10359.16.camel@tardis> Message-ID: <50424F35.2040105@wordshell.net> Hi, Thanks. I used monit, and will code something up of that kind - but I was curious as to whether dovecot had something built in; external monitoring is a somewhat blunter tool, as you don't get to know exactly why there was a problem unless you also parse the logs. Tricky to investigate this kind of issue - it has only happened once, and I had a few hundred people trying to log in, so no time to start debugging! But if it recurs I'll be back to ask for guidance... Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net On 01/09/12 12:15, Noel Butler wrote: > Seen similar, we use mon? for network monitoring, our mon pop3 script > is modified to not only connect, but login, if it cant the trigger > becomes active and if fails again at next check, it alerts engineers > by sms. > > Though, I would investigate the core issue, Timo "apparently" fixed > that hung session stuff in 2.0, it was common in 0.x and 1.x series. > > Cheers > > 1: ( > http://sourceforge.net/projects/mon/files/mon/ ) > > > From ml at smtp.fakessh.eu Sun Sep 2 04:59:42 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 02 Sep 2012 03:59:42 +0200 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler Message-ID: <1346551182.14379.1.camel@localhost> Hi Timo in the present work would be possible to walk to another folder for deleted_to_trash please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From me at junc.org Sun Sep 2 08:30:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 07:30:40 +0200 Subject: [Dovecot] =?utf-8?q?How_Converting/Using_courier-userdb_Files_=3F?= In-Reply-To: <5040C15D.4000303@eurodata.de> References: <5040C15D.4000303@eurodata.de> Message-ID: <421fa10c33e339153cb5532da61e57b2@junc.org> Den 2012-08-31 15:51, Dirk Caspari skrev: > how can i convert/use the Original userdb-Files from courier > Installation ? > The files are looking like a "Special"-DB-Format and are not > readable. it can be added if gdbm is supported in dovecot auth backedn, even convert the gdbm db to sqlite will be suported in shells, when its converted lets say to sqlite then its a matter of define how dovecot use it its a one time run From tss at iki.fi Sun Sep 2 17:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 17:28:28 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120831084233.GE3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: On 31.8.2012, at 11.42, Jost Krieger wrote: > One of our users likes to delete attachments from his mails. He is using > roundcube and we installed a plugin for that. Does the plugin change the mbox file directly? > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error. From eddy.ilg at univation.de Sun Sep 2 18:28:26 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:28:26 +0200 Subject: [Dovecot] Special Folder Mocking Message-ID: <50437B1A.8000808@univation.de> Dear Dovecot List, probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. A simple question: Isn't it possible to advertise one imap folder with different names, depending on the client? E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? Best regards, Eddy Ilg From tss at iki.fi Sun Sep 2 18:32:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:08 +0300 Subject: [Dovecot] Mailbox aliases Message-ID: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: .. mailbox Sent { auto = create special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent alias_for = Sent } .. If "Sent Messages" doesn't exist: - DELETE Sent works as usual - RENAME Sent works as usual If "Sent Messages" is created: - it's symlinked to Sent - DELETE "Sent messages" deletes the symlink, not the contents - RENAME "Sent messages" isn't allowed - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. Thoughts? The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Sun Sep 2 18:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:37 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> On 2.9.2012, at 18.28, Eddy Ilg|univation wrote: > Dear Dovecot List, > > > probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. I was just in the middle of writing the previous mail about this :) > A simple question: > Isn't it possible to advertise one imap folder with different names, depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? No. From eddy.ilg at univation.de Sun Sep 2 18:50:12 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:50:12 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> Message-ID: <50438034.4060604@univation.de> > I was just in the middle of writing the previous mail about this :) Already thought you replied and changed the subject of my post ;) >> A simple question: >> Isn't it possible to advertise one imap folder with different names, depending on the client? >> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? > No. But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? A similar thing is "imap_client_workarounds = tb-extra-mailbox-sep". If what I suggested is implementable then what speaks against it? If I have some time I might try writing a patch... Or maybe a dovecot developer can implement it, who better knows the code. Best, Eddy From tss at iki.fi Sun Sep 2 19:39:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 19:39:46 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50438034.4060604@univation.de> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> <50438034.4060604@univation.de> Message-ID: On 2.9.2012, at 18.50, Eddy Ilg|univation wrote: >> I was just in the middle of writing the previous mail about this :) > Already thought you replied and changed the subject of my post ;) >>> A simple question: >>> Isn't it possible to advertise one imap folder with different names, depending on the client? >>> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? >> No. > But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? Even theoretically it's not something I want to consider. It would cause more trouble than gain. (Some clients identify themselves with ID extension, many don't (e.g. Outlook). Some clients might be unreliably detected using the IMAP command tags they use, but that could identify other clients wrong. Different localizations of clients use different mailbox names.) From dovecot at knutejohnson.com Sun Sep 2 20:29:17 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:29:17 -0700 Subject: [Dovecot] Strange messages? Message-ID: <5043976D.7040806@knutejohnson.com> The following messages show up in the output of a script that does some backup. This too is new with the upgrade to Ubuntu server 12.04.1 and dovecot 2.0.19. Anybody know what would generate these sorts of messages? Where or what would cause them? WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Thanks, -- Knute Johnson From dovecot at knutejohnson.com Sun Sep 2 20:30:18 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:30:18 -0700 Subject: [Dovecot] New log entries with 2.0.19? In-Reply-To: <5041436E.7000208@knutejohnson.com> References: <5041436E.7000208@knutejohnson.com> Message-ID: <504397AA.7040300@knutejohnson.com> A couple of days later and it is back to normal. Never mind. knute... On 8/31/2012 4:06 PM, Knute Johnson wrote: > I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded > Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) > disconnect in IDLE and inactivity time outs. I'm using only IMAP with > two users. Both users access the server from multiple devices, phone, > tablet and desktop computer. I had previously set the > max_userip_connections to 20 and rarely got the following limits > exceeded message. That's probably being triggered when we are both home > and using all of our devices on the same network. I've now changed it > to 40 and we'll see how that does. > > The other is the IDLE and inactivity timeouts. I don't really > understand how IDLE works and I couldn't find an inactivity timeout > setting to play with. There are at this second 32 imap or imap-login > processes running with only two phones and one desktop running. That's > a few more than before the upgrade but not a lot. > > --------------------- Dovecot Begin ------------------------ > > Dovecot limits exceeded: > max_userip_connections: penny from 216.240.58.138 to > 216.240.58.140: 56 Time(s) > > Dovecot disconnects: > Disconnected in IDLE: 485 Time(s) > Inactivity: 343 Time(s) > no auth attempts: 14 Time(s) > no reason: 21 Time(s) > > Everything seems to be working OK so maybe I should just not worry about > it? > > Thanks, > > knute... > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_anonymous_username = anonymous > auth_cache_negative_ttl = 1 hours > auth_cache_size = 0 > auth_cache_ttl = 1 hours > auth_debug = no > auth_debug_passwords = no > auth_default_realm = > auth_failure_delay = 2 secs > auth_first_valid_uid = 500 > auth_gssapi_hostname = > auth_krb5_keytab = > auth_last_valid_uid = 0 > auth_master_user_separator = > auth_mechanisms = plain login > auth_realms = > auth_socket_path = auth-userdb > auth_ssl_require_client_cert = no > auth_ssl_username_from_cert = no > auth_use_winbind = no > auth_username_chars = > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ > auth_username_format = > auth_username_translation = > auth_verbose = no > auth_verbose_passwords = no > auth_winbind_helper_path = /usr/bin/ntlm_auth > auth_worker_max_count = 30 > base_dir = /var/run/dovecot > config_cache_size = 1 M > debug_log_path = > default_client_limit = 1000 > default_idle_kill = 60 > default_internal_user = dovecot > default_login_user = dovenull > default_process_limit = 100 > default_vsz_limit = 256 M > deliver_log_format = msgid=%m: %$ > dict_db_config = > director_doveadm_port = 0 > director_mail_servers = > director_servers = > director_user_expire = 15 mins > disable_plaintext_auth = yes > dotlock_use_excl = yes > doveadm_allowed_commands = > doveadm_password = > doveadm_proxy_port = 0 > doveadm_socket_path = doveadm-server > doveadm_worker_count = 0 > first_valid_gid = 1 > first_valid_uid = 500 > hostname = > imap_capability = > imap_client_workarounds = > imap_id_log = > imap_id_send = > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > import_environment = TZ > info_log_path = > instance_name = dovecot > last_valid_gid = 0 > last_valid_uid = 0 > lda_mailbox_autocreate = no > lda_mailbox_autosubscribe = no > lda_original_recipient_header = > libexec_dir = /usr/lib/dovecot > listen = *, :: > lmtp_proxy = no > lmtp_save_to_detail_mailbox = no > lock_method = fcntl > log_path = syslog > log_timestamp = "%b %d %H:%M:%S " > login_access_sockets = > login_greeting = Dovecot ready. > login_log_format = %$: %s > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > login_trusted_networks = > mail_access_groups = > mail_attachment_dir = > mail_attachment_fs = sis posix > mail_attachment_hash = %{sha1} > mail_attachment_min_size = 128 k > mail_cache_fields = flags > mail_cache_min_mail_count = 0 > mail_chroot = > mail_debug = no > mail_fsync = optimized > mail_full_filesystem_access = no > mail_gid = > mail_home = > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_log_prefix = "%s(%u): " > mail_max_keyword_length = 50 > mail_max_lock_timeout = 0 > mail_max_userip_connections = 10 > mail_never_cache_fields = imap.envelope > mail_nfs_index = no > mail_nfs_storage = no > mail_plugin_dir = /usr/lib/dovecot/modules > mail_plugins = > mail_privileged_group = > mail_save_crlf = no > mail_temp_dir = /tmp > mail_uid = > mailbox_idle_check_interval = 30 secs > mailbox_list_index_disable = no > maildir_copy_with_hardlinks = yes > maildir_stat_dirs = no > maildir_very_dirty_syncs = no > master_user_separator = > mbox_dirty_syncs = yes > mbox_dotlock_change_timeout = 2 mins > mbox_lazy_writes = yes > mbox_lock_timeout = 5 mins > mbox_min_index_size = 0 > mbox_read_locks = fcntl > mbox_very_dirty_syncs = no > mbox_write_locks = dotlock fcntl > mdbox_preallocate_space = no > mdbox_rotate_interval = 0 > mdbox_rotate_size = 2 M > mmap_disable = no > passdb { > args = > deny = no > driver = pam > master = no > pass = no > } > pop3_client_workarounds = > pop3_enable_last = no > pop3_fast_size_lookups = no > pop3_lock_session = no > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_save_uidl = no > pop3_uidl_format = %08Xu%08Xv > postmaster_address = > protocols = " imap" > quota_full_tempfail = no > recipient_delimiter = + > rejection_reason = Your message to <%t> was automatically rejected:%n%r > rejection_subject = Rejected: %s > sendmail_path = /usr/sbin/sendmail > service anvil { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = anvil > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 1 > protocol = > service_count = 0 > type = anvil > unix_listener anvil-auth-penalty { > group = > mode = 0600 > user = > } > unix_listener anvil { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service auth-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = auth -w > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener auth-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service auth { > chroot = > client_limit = 4096 > drop_priv_before_exec = no > executable = auth > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener auth-client { > group = > mode = 0600 > user = > } > unix_listener auth-login { > group = > mode = 0600 > user = $default_internal_user > } > unix_listener auth-master { > group = > mode = 0600 > user = > } > unix_listener auth-userdb { > group = > mode = 0600 > user = > } > unix_listener login/login { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service config { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = config > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = config > unix_listener config { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service dict { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dict > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dict { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service director { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = director > extra_groups = > fifo_listener login/proxy-notify { > group = > mode = 00 > user = > } > group = > idle_kill = 4294967295 secs > inet_listener { > address = > port = 0 > ssl = no > } > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener director-admin { > group = > mode = 0600 > user = > } > unix_listener director-userdb { > group = > mode = 0600 > user = > } > unix_listener login/director { > group = > mode = 00 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service dns_client { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dns-client > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dns-client { > group = > mode = 0666 > user = > } > unix_listener login/dns-client { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service doveadm { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = doveadm-server > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener doveadm-server { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service imap-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = imap-login > extra_groups = > group = > idle_kill = 0 > inet_listener imap { > address = > port = 143 > ssl = no > } > inet_listener imaps { > address = > port = 993 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service imap { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = > unix_listener login/imap { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ipc { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = ipc > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener ipc { > group = > mode = 0600 > user = > } > unix_listener login/ipc-proxy { > group = > mode = 0600 > user = $default_login_user > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service lmtp { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = lmtp > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = lmtp > service_count = 0 > type = > unix_listener lmtp { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service log { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = log > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = log > user = > vsz_limit = 18446744073709551615 B > } > service pop3-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = pop3-login > extra_groups = > group = > idle_kill = 0 > inet_listener pop3 { > address = > port = 0 > ssl = no > } > inet_listener pop3s { > address = > port = 0 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service pop3 { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = pop3 > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = > unix_listener login/pop3 { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ssl-params { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = ssl-params > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = startup > unix_listener login/ssl-params { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > shutdown_clients = yes > ssl = yes > ssl_ca = > ssl_cert = ssl_cert_username_field = commonName > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > ssl_client_cert = > ssl_client_key = > ssl_key = ssl_key_password = > ssl_parameters_regenerate = 168 > ssl_verify_client_cert = no > submission_host = > syslog_facility = mail > userdb { > args = > driver = passwd > } > valid_chroot_dirs = > verbose_proctitle = no > verbose_ssl = no > version_ignore = no > protocol imap { > imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID > ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND > UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 ESEARCH > ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS > mail_max_userip_connections = 40 > } > -- Knute Johnson From p at state-of-mind.de Sun Sep 2 23:32:24 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sun, 2 Sep 2012 22:32:24 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <20120902203221.GB18740@state-of-mind.de> Eddy, * Eddy Ilg|univation : > Dear Dovecot List, > > > probably everyone knows the problem that different clients use > different names for "Sent", "Trash" and so on. > > A simple question: > Isn't it possible to advertise one imap folder with different names, > depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent > Elements" (translated from german) and if the client is Thunderbird, > show the "Sent" folder as "Sent"? the answer is the IMAP extension "SPECIAL USE". We sponsored SPECIAL USE in Dovecot and in Thunderbird. Timo implemented it in Dovecot autumn 2011 and Ben Bucksch added SPECIAL USE support in Mozilla shortly after. Sadly Mozilla has decided to turn Thunderbird down - i.e. hand it over to some 'community' - and I am not sure our contribution it will be released as Mozilla claims they will not add any new features anymore. IF they would you could tell Dovecot to name the SPECIAL USE folders like Outlook expects them and Thunderbird to map its folder on top. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Mon Sep 3 00:01:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 02 Sep 2012 23:01:16 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <5043C91C.2070608@schetterer.org> Am 02.09.2012 17:32, schrieb Timo Sirainen: > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. Hi Timo, funny, yesterday i had problems with this, i am using virtual plugin setup with pop3 also disallow i.e imap Sent/Trash/Drafts download via imap, but allow Junk imap folder to be downloaded via pop3 cause i have a global sieve rule storing, that spam tagged mail is going direct in Junk folder, that usally cant be overided by users. This should avoid redirecting spam tagged mail via sieve by users in prime. This works nice. My users use all kind of os and mail clients typical ISP Setup Default webmail Horde Imp layout is configured for thunderbirds default imap folder layout, also thunderbird is announced to be the supported mail client via the faq help site, not because its the best client appearing, only for ,that it is installable in all major os, so debug is possible widly In the help site and pictured welcome pdf mail there are setups shown for all major mail clients, as outlook, thunderbird , winmail. Not for Apple cause i havent got the chance to make screenshots from it yet. I also have active sync via z-push so setup in android mail app is on the help site too. So far so good. With Problems that i.e outlook is very different in handling imap folder setup through its versions As more and more people going to use mobile, stuff get more complicated As one user uses k9mail android vers 4.11 ( this is recent ), which cant do imap special use yet, so a i.e german Sent Folder ( Gesendet ) is created auto by the k9 client, same user does pop3 via outlook so the problem appeared that he send mail with copy in the "Gesendet" from K9 mail android folder ,but this Folder wasnt disallowed in the pop3 virtual layout , so a loop appeared. I had now to disallow folder "Gesendet/Papierkorb/Entw?rfe" for pop3. Upcomming Versions K9 Mail 4.2 can do imap special use, and it works by testing Only small real World example. I investigated little, and found neither Thunderbird Outlook or Apple is supporting imap special use yet ( seems its on the road for thunderbird ), perhaps someone knows more about this. My meaning to this theme is, its simply teribble for 21 century mail that there is no recommended default imap folder layout for all mailclients, imap special use looks promising , hopefully it will wide spread Goal: Configure mail should be done auto only with emailadress and password including default imap folder layout and match trough all major mail imap clients. So example wiki site for configure dovecot relate to major imap clients and their versions behave in default imap folder layout and support of imapspecial use would be extrem helpfull, in fact yesterday i searched about that. -- Mfg Best Regards Robert Schetterer From me at junc.org Mon Sep 3 00:22:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 23:22:40 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> Message-ID: <0fa581317d82ab7f071329317125bc55@junc.org> Den 2012-08-24 07:18, Voytek Eymont skrev: > I want to offer 'plain' unsecure pop/imap access, just for temp. > testing purpose, do I simply comment out > ssl = required plain is not related to ssl enabled, here i have disabled pop3 imap, so now here i only supports ssl / tls connections better set listen to localhost or another rfc1918 ip From h.reindl at thelounge.net Mon Sep 3 00:32:34 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 02 Sep 2012 23:32:34 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <5043C91C.2070608@schetterer.org> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> <5043C91C.2070608@schetterer.org> Message-ID: <5043D072.4020604@thelounge.net> Am 02.09.2012 23:01, schrieb Robert Schetterer: > Goal: > Configure mail should be done auto only with emailadress and password > including default imap folder layout and match trough all major mail > imap clients. > > So example wiki site for configure dovecot relate to major imap clients > and their versions behave in default imap folder layout and support of > imapspecial use would be extrem helpfull, in fact yesterday i searched > about that. the real problem are stupid clients at all the folders just could called "Sent" on the server and the client can display whatever it wants, this even works if you take some minutes to configure all your clients to use the same folder and the apple is coming: eahc different version of their clients is creating different sent folders - after update MacOSX it happens sometimes that it creates a new incarnation on the servr and spit new messages in the new one instead the over months used existing and if a users owns different apple-devices he ends up in many incarnations * Sent * Sent Messages * Sent Messages (acount name) * Gesendet and this is only the example of ONE apple-only-user you can imagine what happens using different other clients and smartphone-types this is nothing that can be fixed on the server side the idiotic client developers should take care of the different possible existing special folders and use any of them if existing instead create a new one -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 01:31:11 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:31:11 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0fa581317d82ab7f071329317125bc55@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> Message-ID: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From raubvogel at gmail.com Mon Sep 3 01:38:53 2012 From: raubvogel at gmail.com (Mauricio) Date: Sun, 2 Sep 2012 22:38:53 +0000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> Message-ID: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> I thought you could specify ssl to be wanted but not required, but I might be thinking on client side. Sent via iPhone -----Original Message----- From: "Voytek Eymont" Sender: dovecot-bounces at dovecot.orgDate: Mon, 3 Sep 2012 08:31:11 To: Subject: Re: [Dovecot] Disable security for testing? On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From voytek at sbt.net.au Mon Sep 3 01:44:16 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:44:16 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <710bc96f1513f58dd51935702626803e.squirrel@sbt.net.au> On Mon, September 3, 2012 8:38 am, Mauricio wrote: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. Sent via iPhone I'm happy to have it as a default requirement, seems like a reasonable requirement, this was just for some temp testing/verification etc -- Voytek From me at junc.org Mon Sep 3 02:26:20 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:26:20 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <0535537af33849715f4b3efd97569d4d@junc.org> Den 2012-09-03 00:38, Mauricio skrev: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. the point is ? > Sent via iPhone i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port 110/143 anymore From h.reindl at thelounge.net Mon Sep 3 02:32:57 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 01:32:57 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0535537af33849715f4b3efd97569d4d@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> Message-ID: <5043ECA9.5010005@thelounge.net> Am 03.09.2012 01:26, schrieb Benny Pedersen: > i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port > 110/143 anymore why? STARTTLS is 110/143 per definition! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From me at junc.org Mon Sep 3 02:40:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:40:56 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <5043ECA9.5010005@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: Den 2012-09-03 01:32, Reindl Harald skrev: > STARTTLS is 110/143 per definition! EOD From adi at ddns.com.au Mon Sep 3 02:41:14 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 09:41:14 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> Message-ID: <20120903094114.2a75835d@adi.ddns.local> On Fri, 31 Aug 2012 17:11:07 +0300 Timo Sirainen wrote: > > The POP/IMAP part is working fine. What I'm trying to do is to use > > Dovecot SASL implementation in Postfix to do SMTP authentication in > > a similar manner. The problem I have with my current configuration > > is that SMTP authentication succeeds if only the username matches, > > because password forwarding works if the authentication succeeds > > with any given password, as documented at > > http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy > > Dovecot has no SMTP proxy (currently). And anyway Postfix doesn't use > SMTP to do authentication, Postfix authenticates using Dovecot's > internal protocol, which replies that Postfix should do the proxying, > which it of course doesn't do. Yes, I know and that's exactly what I was trying to do: use Dovecot authentication method in Postfix to authenticate the user. Because of the fact that Dovecot doesn't do SMTP authentication, I was thinking of a way of using its authentication service by getting the SMTP login credentials from the backend POP/IMAP server. The request may sound a bit unusual, but in our case it makes sense. For our setup we currently run: - a farm of backend SMTP/POP/IMAP servers that are hosting the mailboxes and where the user credentials are managed. They are running Courier IMAP. - a group of SMTP/POP/IMAP proxies. These proxies are currently replicating the login credentials from the backend servers and the routing to the backends using a local database. Perdition is currently the POP/IMAP proxy, but having it replaced with Dovecot would help us in getting the password forwarding to the backends running, which means we wouldn't need to store the credentials on the proxy, only the user->host routing entries. Are there any plans to have Dovecot authentication service to do SMTP authentication against IMAP or POP3 proxy provided information? This, of course, means we'd have the authentication result tied to the response of the backend IMAP/pop3 server. > > My question is, given the above: is there a way to get SMTP > > authentication properly in this scenario? > > Make Postfix authenticate against the backend Dovecot server. You'll > need to setup service auth { inet_listener } to some port for it. The POP/IMAP backends are running Courier IMAP, as I've just mentioned and due to the existing hosting environment it's very unlikely to replace it with something else. -- Adi Pircalabu, System Administrator Discount Domain Name Services Pty Ltd, a Total Internet Company PO Box 887, Hawthorn Vic 3122, Australia, T +61 3 9815 6868 Ask me about cloud hosting services From tss at iki.fi Mon Sep 3 02:47:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 02:47:48 +0300 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <20120903094114.2a75835d@adi.ddns.local> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: On 3.9.2012, at 2.41, Adi Pircalabu wrote: > Are there any plans to have Dovecot authentication service to do SMTP > authentication against IMAP or POP3 proxy provided information? This, > of course, means we'd have the authentication result tied to the > response of the backend IMAP/pop3 server. I don't see any clean way of doing that. >>> My question is, given the above: is there a way to get SMTP >>> authentication properly in this scenario? >> >> Make Postfix authenticate against the backend Dovecot server. You'll >> need to setup service auth { inet_listener } to some port for it. > > The POP/IMAP backends are running Courier IMAP, as I've just mentioned > and due to the existing hosting environment it's very unlikely to > replace it with something else. Maybe use IMAP authentication as the backend? pam_imap at least can do that. Or you can already also use Dovecot v2.1's passdb imap to do this, pretty much equivalent to pam_imap. From ml at smtp.fakessh.eu Mon Sep 3 02:55:20 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 01:55:20 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: <32bbc94eb73b728d0a4646c6d7a3b67d@smtp.fakessh.eu> I currently work this plugin with several folders nice a documentation include Le 2012-09-02 03:59, ml a ?crit?: > Hi Timo > > in the present work would be possible to walk to another folder for > deleted_to_trash > > please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From me at junc.org Mon Sep 3 02:55:21 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:55:21 +0200 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Den 2012-09-03 01:47, Timo Sirainen skrev: > Maybe use IMAP authentication as the backend? pam_imap at least can > do that. Or you can already also use Dovecot v2.1's passdb imap to do > this, pretty much equivalent to pam_imap. drop pam, and use saslauthd with remote imap, or setup cyrus sasl with sql/ldap/whatever one needs, just in case one more ask why i did not use dovecot :) From tss at iki.fi Mon Sep 3 03:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 03:18:22 +0300 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: On 2.9.2012, at 4.59, ml wrote: > in the present work would be possible to walk to another folder for > deleted_to_trash I suppose, with some code modifications. But I'd prefer if nobody used this plugin (or anything equivalent to its functionality). From ml at smtp.fakessh.eu Mon Sep 3 03:26:54 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 02:26:54 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: References: <1346551182.14379.1.camel@localhost> Message-ID: <303d5f8de294c07f547603b58a4aec85@smtp.fakessh.eu> Le 2012-09-03 02:18, Timo Sirainen a ?crit?: > On 2.9.2012, at 4.59, ml wrote: > >> in the present work would be possible to walk to another folder for >> deleted_to_trash > > I suppose, with some code modifications. But I'd prefer if nobody > used this plugin (or anything equivalent to its functionality). until the it works very well as it is with an inclusion in dovecot.conf. and why not use it I have folders in my email that I am happy with this empty nice plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From adi at ddns.com.au Mon Sep 3 04:55:20 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 11:55:20 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Message-ID: <20120903115520.218c8190@adi.ddns.local> On Mon, 03 Sep 2012 01:55:21 +0200 Benny Pedersen wrote: > Den 2012-09-03 01:47, Timo Sirainen skrev: > > > Maybe use IMAP authentication as the backend? pam_imap at least can > > do that. Or you can already also use Dovecot v2.1's passdb imap to > > do this, pretty much equivalent to pam_imap. I had a look at pam_imap and had to fix the spec file to get it built for CentOS 6 64bit. For the record, I had to add the following in the %build section: export CFLAGS="%{optflags} -fPIC" export CXXFLAGS="%{optflags} -fPIC" However: > drop pam, and use saslauthd with remote imap, or setup cyrus sasl > with sql/ldap/whatever one needs Yep, saslauthd with rimap appears to be exactly what I need. > just in case one more ask why i did not use dovecot :) The password forwarding feature is one of the reasons I started looking into Dovecot. Thanks to you both for your help. -- Adi Pircalabu, System Administrator From amateo at um.es Mon Sep 3 09:48:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 08:48:54 +0200 Subject: [Dovecot] exclude a folder for lazy expunge (move to trash) In-Reply-To: <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> References: <503491C4.7030208@um.es> <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> Message-ID: <504452D6.8@um.es> El 31/08/12 15:23, Timo Sirainen escribi?: > On 22.8.2012, at 11.01, Angel L. Mateo wrote: > >> I'm configuring lazy expunge plugin to act as a kind of backup of my user's emails. >> >> I'm getting a little problem because a lot of my users use "move to trash" behaviour in their email client (most of them are using thunderbird which has no option to hide deleted messages and is a hassle working with mark as deleted with it). The problem combining move to trash with lazy expunge is that messages are stored in the expunge namespace twice, first when it was deleted from the original folder and the second when is deleted from the trash. >> >> So... is there any way to exclude trash folder from lazy expunge plugin, so mails are stored in the expunge namespace only when they are deleted from their original folder and not from the trash? > > Not right now. Maybe it could be automatically done when SPECIAL-USE is enabled and the deletion is from a \Trash mailbox. > >> Another idea... We are purging trash folders with a cron running "doveadm purge" commands. Is there any way to run this command disabling lazy expunge plugin? > > doveadm purge shouldn't affect lazy expunge, but you can disable lazy_expunge by using something like doveadm -o mail_plugins=everything,except,lazy_expunge expunge .. > Sorry, I meant "doveadm expunge" (I always confuse these two commands) which does affect lazy expunge -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From alec at alec.pl Mon Sep 3 10:02:13 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 03 Sep 2012 09:02:13 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <504455F5.7000504@alec.pl> On 09/02/2012 05:32 PM, Timo Sirainen wrote: > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > Thoughts? Yes. It should solve some problems, but I think that only wide-use of SPECIAL-USE is a solution for this. Also.... I don't use current version of dovecot, but from reading this list I remember that SPECIAL-USE implementation in dovecot is read-only. What I'd like to see is possibility to set special-use flags by the client - no server configuration required. It means, when user connects to his mailbox for the first time and special folders doesn't exist, the client (supporting SPECIAL-USE) can create folders according to its configuration and set their special-use flags. Or at any time when it find that special folder doesn't exist. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From zucca at systemschmiede.com Mon Sep 3 10:01:51 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Mon, 03 Sep 2012 09:01:51 +0200 Subject: [Dovecot] Imap Ghost folder In-Reply-To: <292te8ob8kv8@mids.svenhartge.de> References: <50341D40.9090001@systemschmiede.com> <50353D32.2080401@systemschmiede.com> <5038B2C5.3090408@systemschmiede.com> <4CE233FA-2954-4C39-9F79-13C12ADFDCF2@iki.fi> <5038CB71.5010808@systemschmiede.com> <503C818A.4010203@systemschmiede.com> <503DE2D9.3080300@systemschmiede.com> <20120829204814.GA9265@daniel.localdomain> <503F0B77.5030206@systemschmiede.com> <504072FA.8020606@systemschmiede.com> <56D9DD4B-ED96-47C6-96F4-322D6907508C@iki.fi> <5040C32C.2020706@systemschmiede.com> <85E5DEFA-D38E-441E-BFE9-64FC820069E8@iki.fi> <5040C789.8000502@systemschmiede.com> <292te8ob8kv8@mids.svenhartge.de> Message-ID: <504455DF.4020202@systemschmiede.com> >> hm, but 2.1.9 is not in backports yet, is it? > And will not be for some time. 2.1.9 is not even in Unstable and even if > it were, it would not migrate to testing (and be an eligible backporting > target) due to the freeze before the Wheezy release. So, no workaround for this? Noone can reproduce it using my configuration? Very strange.. Thanks Sascha. From amateo at um.es Mon Sep 3 11:41:24 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 10:41:24 +0200 Subject: [Dovecot] doveadm command to block a user? Message-ID: <50446D34.5000702@um.es> Hello, Is there any command to be able to lock imap access to a specific user? With doveadm kick I could close his connections, but I want to avoid future ones too and just for imap. Any way to do it? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tseveendorj at gmail.com Mon Sep 3 12:10:14 2012 From: tseveendorj at gmail.com (tseveendorj) Date: Mon, 03 Sep 2012 17:10:14 +0800 Subject: [Dovecot] dovecot sasl + postfix issue Message-ID: <504473F6.5010205@gmail.com> Hello, I'm trying to configure postfix + Dovecot SASL for user authenticated mail relay. I set following configuration on postfix queue_directory = /var/spool/postfix smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_authenticated_header = yes smtpd_sasl_security_options = noanonymous smtpd_sasl_local_domain = $myhostname broken_sasl_auth_clients = yes smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination Following are dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_mechanisms = plain login digest-md5 base_dir = /var/run/dovecot/ mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r when I see telnet localhost 25 eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 beastie ESMTP Postfix (Ubuntu) ehlo beastie 250-beastie 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN quit 221 2.0.0 Bye Connection closed by foreign host. there is no 250-AUTH How do I fix this ? From Bill at knoxvillechristian.org Mon Sep 3 12:36:32 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Mon, 03 Sep 2012 05:36:32 -0400 Subject: [Dovecot] dovecot sasl + postfix issue In-Reply-To: <504473F6.5010205@gmail.com> References: <504473F6.5010205@gmail.com> Message-ID: <50447A20.7060900@knoxvillechristian.org> You should get AUTH after you STARTTLS. Bill On 9/3/2012 5:10 AM, tseveendorj wrote: > Hello, > > I'm trying to configure postfix + Dovecot SASL for user authenticated > mail relay. > > I set following configuration on postfix > > queue_directory = /var/spool/postfix > smtpd_sasl_auth_enable = yes > smtpd_sasl_type = dovecot > smtpd_sasl_path = private/auth > smtpd_sasl_authenticated_header = yes > smtpd_sasl_security_options = noanonymous > smtpd_sasl_local_domain = $myhostname > broken_sasl_auth_clients = yes > smtpd_recipient_restrictions = reject_unknown_sender_domain, > reject_unknown_recipient_domain, reject_unauth_pipelining, > permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination > > Following are dovecot -n > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_mechanisms = plain login digest-md5 > base_dir = /var/run/dovecot/ > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_cipher_list = > ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to <%t> was automatically rejected:%n%r > > > > when I see telnet localhost 25 > > eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > 220 beastie ESMTP Postfix (Ubuntu) > ehlo beastie > 250-beastie > 250-PIPELINING > 250-SIZE 10240000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > quit > 221 2.0.0 Bye > Connection closed by foreign host. > > there is no 250-AUTH > > How do I fix this ? From h.reindl at thelounge.net Mon Sep 3 13:05:10 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:05:10 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: <504480D6.1000900@thelounge.net> Am 03.09.2012 01:40, schrieb Benny Pedersen: > Den 2012-09-03 01:32, Reindl Harald skrev: > >> STARTTLS is 110/143 per definition! > > EOD oh yeah this is the right attitude * open thunderbird * go to server settings * switch between STARTTLS/SSL * look how the default port changes for SMTP port 465 is even deprecated and you should use 587 (submission) with STATTLS for endusers but do what you want if you like support-calls as example from customers sitting in a network where most outgoing ports are blocked (including 993 and 995) while 143 is open, yes i have seen this from customers sitting in gonvernment networks but hey, for you this is EOD because you know it better SMTP STARTTLS: 587 SMTP over SSL: 465 POP3 STARTTLS: 110 POP3 over SSL: 995 IMAP STARTTLS: 143 IMAP over SSL: 993 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 13:15:27 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 20:15:27 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <504480D6.1000900@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> Message-ID: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> > but hey, for you this is EOD because you know it better > > SMTP STARTTLS: 587 > SMTP over SSL: 465 > POP3 STARTTLS: 110 > POP3 over SSL: 995 > IMAP STARTTLS: 143 > IMAP over SSL: 993 in my limited experience, when I tried to set Outlook Express clients with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got it on 993) question: is it because Outlook Express does SSL only, but not TLS? (or is it simply because it's crappy application ?) thanks, enquiring minds need to know, -- Voytek From h.reindl at thelounge.net Mon Sep 3 13:19:16 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:19:16 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <50448424.7000804@thelounge.net> Am 03.09.2012 12:15, schrieb Voytek Eymont: >> but hey, for you this is EOD because you know it better >> >> SMTP STARTTLS: 587 >> SMTP over SSL: 465 >> POP3 STARTTLS: 110 >> POP3 over SSL: 995 >> IMAP STARTTLS: 143 >> IMAP over SSL: 993 > > > in my limited experience, when I tried to set Outlook Express clients with > my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; > (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got > it on 993) it's a microsoft problem > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) because it is a crappy application you see htunderbird perfectly switching between STARTTLS/SSL ports you can configure each webmail i know to use STARTTLS you can configure even Apple Mail for both usecases -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From amateo at um.es Mon Sep 3 14:16:35 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 13:16:35 +0200 Subject: [Dovecot] Syntax for doveadm auth cache Message-ID: <50449193.8080101@um.es> Hello, What is the correct syntax of doveadm auth cache command? According to command line help: usage: doveadm [-Dv] [-f ] [] altmove [-u |-A] [-S ] [-r] auth [-a ] [-x ] [] auth cache but when I try to run doveadm auth cache, then I'm asked for a password, so I think the command is trying to authenticate cache user. Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): doveadm auth cache flush 4 cache entries flushed So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From sven at svenhartge.de Mon Sep 3 14:31:18 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 3 Sep 2012 13:31:18 +0200 Subject: [Dovecot] finding messages deleted before timespec Message-ID: <4934p48so7v8@mids.svenhartge.de> Hi! I've been pondering about doveadm-search-query(7) and trying to formulate a search query for some time now and it seems there is no way to search for messages, which have been marked DELETED before a certain time. I want to use this to expunge all DELETED messages which have been in that state for 24h or longer. For example a cronjob runs at midnight to expunge those messages but a user deletes one just 10 minutes before that, this message should not be deleted until the day after. This is my first time diving deeper into IMAP search queries so I have the nagging feeling I am missing something important here. Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel at dhag.net Mon Sep 3 16:32:55 2012 From: daniel at dhag.net (Daniel Funke) Date: Mon, 03 Sep 2012 15:32:55 +0200 Subject: [Dovecot] dovecot proxy ssl-parameter from database? Message-ID: <5044B187.6000609@dhag.net> Hi List, I want to migrate my IMAP-users from an old Courier-IMAP-Server to a new dovecot-Server. The old server supports IMAP without and with TLS. The new one only with TLS. To users should be migrated one by one with changing some client parameters. To leave the servername of the imap server I want to use the dovecot proxy during migration. the proxy uses a sql db to look for the old or new host for every user. So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) I tried with http://wiki2.dovecot.org/UserDatabase/ExtraFields and returned a field "userdb_ssl" ("required" for host dovecot, "yes" for host courier). But it doesn't work. A migrated user can still login without tls to the proxy and the dovecot backend. Any ideas? Daniel Here is my doveconf -n # 2.1.9: /opt/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login default_process_limit = 500 disable_plaintext_auth = no listen = * managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /opt/dovecot/etc/dovecot/sql.conf driver = sql } protocols = imap pop3 service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3s { port = 0 } } ssl_cert = References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <20120903160119.06de7fcd@zetkin.int.primekey.se> On Mon, 3 Sep 2012 20:15:27 +1000 "Voytek Eymont" wrote: > in my limited experience, when I tried to set Outlook Express clients > with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on > 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on > 143, but, got it on 993) > > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) The ports 143 and 993 are used in different way when you want to have TLS. In case of 143, you begin the communication in unencrypted mode, then tell the server that you want to initiate a TLS handshake (you literally send a STARTTLS string to it). After that both client and server start negotiating the TLS connection (and hopefully they manage to do it :). With the port 993 the expected/standard behavior is that you begin the TLS handshake right upon establishing the connection. Normally the servers will not accept any unencrypted IMAP command on it (in other words, you don't send STARTTLS to the server, the server expects you to start the handshake right away). Similar concept is implemented for POP3, SMTP, LDAP, XMPP, and many others... Btw, SSL and TLS are kind of the same things (or to be more precise when people are talking about it they refer to the same thing, unless they're crypto techies which are using the terminology correctly :) Best regards P.S. It's quite probably a crappy application as well :) -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From Jost.Krieger+dovecot at rub.de Mon Sep 3 17:11:21 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 3 Sep 2012 16:11:21 +0200 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: <20120903141121.GR3002@ruhr-uni-bochum.de> [Sorry about the personal reply.] On Sun, Sep 02, 2012 at 05:28:28PM +0300, Timo Sirainen wrote: > > On 31.8.2012, at 11.42, Jost Krieger wrote: > > > One of our users likes to delete attachments from his mails. He is using > > roundcube and we installed a plugin for that. > > Does the plugin change the mbox file directly? No, it just uses IMAP. > > > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) > > This looks like it does. Existing emails must not be changed, IMAP > protocol forbids it (and Dovecot internals don't support it either). > The only safe way to drop attachments is to assign a new UID for the > mail, which basically means saving a new message without attachments > and deleting the old one. The Content-Length header doesn't really > have anything to do with this error. Sorry if I was unclear. This is exactly what the plugin does. It builds a new message, starting with the old message and leaving out the unwanted attchment(s). Then it saves the new message, deletes the old message and finally copies over the flags. I had a look at the handling of "Content-Length" in dovecot and had the impression that a new header was added but the old was not removed (if formally ok) and on reading the first header wins. But I'm not sure, of course. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From kgc at corp.sonic.net Mon Sep 3 21:26:55 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:26:55 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected Message-ID: <5044F66F.3010505@corp.sonic.net> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) I've dropped one of the servers out of the ring to prevent suffering but this is a less than ideal situation. Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 10000 director_mail_servers = 1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 director_servers = a.director.foo b.director.foo disable_plaintext_auth = no doveadm_proxy_port = 1842 login_trusted_networks = 10.0.0.1 mbox_write_locks = fcntl passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 20103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_ca = Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:23:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla2 host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:23:32 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:24:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:05 a.director. a dovecot: director: Warning: Delaying new user requests until ring is synced >Sep 3 09:24:32 a.director. a dovecot: director: Warning: Ring is synced, continuing delayed requests >Sep 3 09:24:41 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:24:41 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:41 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:24:41 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:25:11 b.director. b dovecot: director: Error: User hash 2285697953 is being redirected to two hosts: 10.10.10.39 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689467) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 623192092 is being redirected to two hosts: 10.10.10.76 and 10.10.10.39 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 1683990717 is being redirected to two hosts: 10.10.10.43 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:25:12 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director lef -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From robert at schetterer.org Mon Sep 3 21:32:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 03 Sep 2012 20:32:59 +0200 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <5044F7DB.9080601@schetterer.org> Am 03.09.2012 20:26, schrieb Kelsey Cummings: > I've had 2x director ring up and running with production load on 2.1.8 > with around 10,000 active connections for two weeks and everything has > been working great - until this morning. meanwhile try upgrade to 2.1.9 i cant see big changes there for director which should brake something http://hg.dovecot.org/dovecot-2.1/rev/bc86680293d2 however its better debug with latest patchlevel -- Best Regards MfG Robert Schetterer From eddy.ilg at univation.de Mon Sep 3 21:36:22 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Mon, 03 Sep 2012 20:36:22 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. Message-ID: <5044F8A6.7060200@univation.de> Hi, In the documentation of dovecot-antispam it is mentioned that moving mails out of the spam folders trains them as non-spam. I.e. thunderbird automatically deletes old spam messages (after a time limit, i.e. 14 days). Will this also train the old spam messages as non-spam? Cheers, Eddy From kgc at corp.sonic.net Mon Sep 3 21:37:57 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:37:57 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F7DB.9080601@schetterer.org> References: <5044F66F.3010505@corp.sonic.net> <5044F7DB.9080601@schetterer.org> Message-ID: <5044F905.60204@corp.sonic.net> On 9/3/2012 11:32 AM, Robert Schetterer wrote: > Am 03.09.2012 20:26, schrieb Kelsey Cummings: >> I've had 2x director ring up and running with production load on 2.1.8 >> with around 10,000 active connections for two weeks and everything has >> been working great - until this morning. > > meanwhile try upgrade to 2.1.9 I did that in the second sentence. ;) One of the servers had a single segfault but this wasn't the original cause and didn't show up until after I was poking. director[30709]: segfault at 40 ip 0000000000405d37 sp 00007fff42af80e0 error 6 in director[400000+f000] -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Mon Sep 3 22:06:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 22:06:49 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. > > There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) Did you try stopping both and then starting them again? That clears up all the state they have. > Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. Directors themselves think they're having trouble connecting to each others.. Annoyingly it doesn't give specific error messages about what happened. I should improve the logging.. If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. From kgc at corp.sonic.net Tue Sep 4 04:58:16 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 18:58:16 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <50456038.4070008@corp.sonic.net> On 9/3/2012 12:06 PM, Timo Sirainen wrote: > Did you try stopping both and then starting them again? That clears up all the state they have. I'm not sure that they were both down when restarting them and will try this tonight. > If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. I can't rule that out but there are not any signs that there are any hardware, OS or network related issues. Thanks for gettting the ring status into doveadm by the way. At least our monitoring caught this quickly. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From bmalik at ihlas.net.tr Tue Sep 4 07:15:23 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Mon, 3 Sep 2012 21:15:23 -0700 (PDT) Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <20120814230403.GA11246@daniel.localdomain> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> Message-ID: <1346732123552-37388.post@n4.nabble.com> i am sorry when i login to the server using pop3s, i see in the dovecot log; @4000000050457dd9179c653c pop3-login: Info: Login: user=, method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, session= if i add my mail address into the users-with-forced-encryption file, i would not reach my account. i get an error; @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script denied access to user test at byserver.net whereas i connect to the account via pop3s if i remove my mail address from the users-with-forced-encryption there is no problem. my postlogin_pop3.sh; #!/usr/local/bin/bash if [ "$SECURED" == "secure" ] then :# handle secure connections here else :# handle insecure connections here if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption then printf "* NO [ALERT] You are not allowed to connect without encryption\r\n" exit 0 fi fi exec "$@" ########### there is also no problem related to SSL certificate what can be the issue ? -- View this message in context: http://dovecot.2317879.n4.nabble.com/about-postlogin-in-dovecot2-tp5993p37388.html Sent from the Dovecot mailing list archive at Nabble.com. From bangkokmaco at gmail.com Tue Sep 4 07:21:54 2012 From: bangkokmaco at gmail.com (cc young) Date: Mon, 3 Sep 2012 21:21:54 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting Message-ID: <1346732514245-37389.post@n4.nabble.com> cannot get TB to recognize either pop3/s or imap/s server can connect just fine with: openssl s_client -connect ms1.myserver.net:993 . login ... but trying with TB /var/log/mail.log gets: dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx -- View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html Sent from the Dovecot mailing list archive at Nabble.com. From me at junc.org Tue Sep 4 08:00:35 2012 From: me at junc.org (Benny Pedersen) Date: Tue, 04 Sep 2012 07:00:35 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. In-Reply-To: <5044F8A6.7060200@univation.de> References: <5044F8A6.7060200@univation.de> Message-ID: Den 2012-09-03 20:36, Eddy Ilg|univation skrev: > In the documentation of dovecot-antispam it is mentioned that moving > mails out of the spam folders trains them as non-spam. I.e. > thunderbird automatically deletes old spam messages (after a time > limit, i.e. 14 days). Will this also train the old spam messages as > non-spam? i hope it depends on dovecot-antispam knows the trashcan or not, but seen from here i think this is really a bug, here i use v1 of dovecot and i dont know if this is changed in v2 dovecot if needs be post plugin config, here i am unsure about this aswell :/ From stsiol at yahoo.co.uk Tue Sep 4 08:59:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 06:59:47 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: cc young > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 7:21 > Subject: [Dovecot] thunderbird not connecting > > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: > http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. > Thunderbird is not the best of clients out there. It's ok but not the best. I am using TB on my sites and found out that it has many issues. Can you be a bit more specific ? Are you trying to connect to Secure IMAP ? (993) on dovecot ? In that case, as far as I remember you have to "fool" Thunderbird during the mail account set up process.? It's a bit crazy, I know but it's true. I have documented everything. Let me know where the hickup is and I'll try to get the docs and let you know how to work around this. HTH, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Sep 4 10:31:22 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 08:31:22 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 9:42 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 12:59 PM, Spyros Tsiolis wrote: > > >> >>----- Original Message ----- >>> From: cc young >>> To: dovecot at dovecot.org >>> Cc: >>> Sent: Tuesday, 4 September 2012, 7:21 >>> Subject: [Dovecot] thunderbird not connecting >>> >>> cannot get TB to recognize either pop3/s or imap/s server >>> >>> can connect just fine with: >>> >>> openssl s_client -connect ms1.myserver.net:993 >>> . login ... >>> >>> but trying with TB /var/log/mail.log gets: >>> >>> dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html >>> Sent from the Dovecot mailing list archive at Nabble.com. >>> >> >> >>Thunderbird is not the best of clients out there. >>It's ok but not the best. >> >>I am using TB on my sites and found out that it has many issues. >>Can you be a bit more specific ? >>Are you trying to connect to Secure IMAP ? (993) on dovecot ? >> >>In that case, as far as I remember you have to "fool" Thunderbird >>during the mail account set up process.? >> >>It's a bit crazy, I know but it's true. I have documented everything. >>Let me know where the hickup is and I'll try to get the docs and >>let you know how to work around this. >> >>HTH, >> > > >right - ??trying to connect to Secure IMAP ? (993) on dovecot > > >not in love with thunderbird, but need to access via linux and ms > > >any help / insight would be wonderful > >? Hi again, Taken from my docs . This is the first part for Mozilla Thunderbird setup. I've seen this work both on Win32 systems and linux systems (mainly Linux Mint). There's a part on pop3/smtp. I am sure you can skip that and do your own magic for those two protocols :? Pre-Installation, Wizard Configuration -------------------------------------- During initial Thunderbird startup, the admin is greeted by a welcome window named ?Mail Account Setup?? Click on Cancel Installation ------------ Click on ?Edit? -> ??Account Settings? New window pops up Click on ?Add? New window pops up Fill in ?Description? (e.g. ?Name Surname mailbox?) On ?Server Name? enter the IP address (e.g. 192.168.3.5) On ?Port? enter the SMTP port (e.g. 25) On ?Security and Authentication? ? ?Connection Security? enter none On ?Authentication Method? choose Password transmitted insecurely On ?Username? enter the users' username including the FQDN (e.g. name at domain.gr) Click on OK From ?Edit?, click on ?Account Settings? From ?Account Actions? click on ?Add Mail Account? New windows pops up Enter in the same name for ?Your name? Enter in the same e-mail address for ?Email address? On ?Password? enter in users' password Click on ?Continue? then without waiting click on ?Manual Config? Once ?Manual Config? is clicked, thunderbird will allow the admin to perform additional configuration tasks For ?Incoming? the admin will choose ?IMAP?, Server Hostname will be set to the servers' IP address (again, 192.168.3.5) From ?Port? choose ?993? From ?SSL? choose ?SSL/TLS? From ?Authentication? choose ?Autodetect? For ?Outgoing? the admin will let the ?SMTP? option For ?Server Hostname?, choose the same IP address (e.g. 192.168.3.5) For ?Port? choose ?25? For ?SSL? choose ?None? For ?Authentication? choose ?Normal Password? On the Username field, enter the full user name, e.g. ?user at domain.gr? Click on ?Re-test? Click on ?Create Account? Check on the ?I understand the Risks? checkbox? Click on ?Create Account? Click on ?Confirm Security Exception? Click on ?OK? Hope this helps and best regards, Spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From bangkokmaco at gmail.com Tue Sep 4 10:50:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Tue, 4 Sep 2012 14:50:40 +0700 Subject: [Dovecot] problems with ssl cert Message-ID: in conf.d/10-ssl.conf: ssl = yes ssl_cert = I upgraded dovecot from 1.1 to 2.1.8 pop3/imap works but i have a problem. i use pop3 protocol on outlook 2010 for my account. and i set keep my mails on the server for 5 days or until i delete them. These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. if i even delete a mail from outlook the server doesn't delete or old mails are not deleted. what can be the problem ? thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394.html Sent from the Dovecot mailing list archive at Nabble.com. From robert at schetterer.org Tue Sep 4 11:30:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 10:30:22 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST Message-ID: <5045BC1E.9040700@schetterer.org> Hi, perhaps somebody wants to this --snip Microsoft? Outlook? 2013 Preview Outlook 2013 Preview implements the IMAP LIST extension specified in [RFC6154] as the XLIST command. ---snip taken out of [MS-STANOIMAP].pdf which zip you may download here http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 11:34:59 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 09:34:59 +0100 (BST) Subject: [Dovecot] problems with ssl cert In-Reply-To: References: Message-ID: <1346747699.13635.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: "cc "maco" young" > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 10:50 > Subject: [Dovecot] problems with ssl cert > > in conf.d/10-ssl.conf: > > ssl = yes > ssl_cert = ssl_key = > when try to set up an account in thrunderbird, get /var/log/mail.log: > > imap-login: Disconnected (no auth attempts): rip=223.205.150.234, > lip=xx.xx.xx.xx, TLS: SSL_read() failed: error:14094416:SSL > routines:SSL3_READ_BYTES:sslv3 alert certificate unknown: SSL alert number > > openssl s_client -connect mail.myserver.com:993 - successful > > using the same cert in gning / chrome is successful, so think the cert is ok > Hi again, Do a fresh Thunderbird installation and get rid of any folders? related to thunderbird on either OS you have installed it on. s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From robert at schetterer.org Tue Sep 4 13:53:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 12:53:33 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045BC1E.9040700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> Message-ID: <5045DDAD.3010705@schetterer.org> Am 04.09.2012 10:30, schrieb Robert Schetterer: > Hi, perhaps somebody wants to this > > --snip > Microsoft? Outlook? 2013 Preview > Outlook 2013 Preview implements the IMAP LIST extension specified in > [RFC6154] as the XLIST command. > ---snip > > taken out of [MS-STANOIMAP].pdf > > which zip you may download here > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > just for info, by small testing Outlook 2013 preview imap specialuse is working with dovecot at minimum for Trash and Sent i have set this mailbox Sent { special_use = \Sent auto=subscribe } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash auto=subscribe } examples are in /etc/dovecot/conf.d/15-mailboxes.conf couldnt test Archive/Drafts/Junk but they "look" Working i had my problems with some new gui functions *g the preview is only in english or spanish so dont know if it will work with other languages then english, lets hope so feel free to test yourself -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 16:31:49 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 14:31:49 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <1346765509.32698.YahooMailNeo@web132201.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 16:00 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 2:31 PM, Spyros Tsiolis wrote: > >> . . . . ?%< . . . . . %< . . . . ?. . . . >? >what you showed is what I remember when I was last hacking in TB. ?now all options are gone. ?screenshot attached > > >think I know problem. ?got tired of TB - no options, no feedback. ?ok if everything works; otherwise sucks. > > >went to claws. ?they have actual messages, actual logs. ?had similar problem, but claws easily allowed me to override. ?see this:?http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2199 > > >know my cert chain ok because?http://www.sslshopper.com/ssl-checker.html checks chain ok > > >anyway, thanks for your help, and goodbye thunderbird > > > > I remember being stuck on this. I know what you are saying. Give it another go. Tinker with it. You'll get there. If you're fed up with TB, that's another thing. However, you _can_ get round this. I know I did. Just my 2c, Regards, spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Sep 4 18:20:08 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 4 Sep 2012 17:20:08 +0200 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <20120904152007.GA20273@PC211.ikt.de> On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: > Hi! [..] > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. http://wiki2.dovecot.org/Tools/Doveadm/Expunge HTH Dennis [..] From alessio at skye.it Tue Sep 4 18:40:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 04 Sep 2012 17:40:33 +0200 Subject: [Dovecot] doveadm command to block a user? In-Reply-To: <50446D34.5000702@um.es> References: <50446D34.5000702@um.es> Message-ID: <504620F1.5040801@skye.it> Il 03/09/2012 10:41, Angel L. Mateo ha scritto: > Hello, > > Is there any command to be able to lock imap access to a specific > user? With doveadm kick I could close his connections, but I want to > avoid future ones too and just for imap. > > Any way to do it? > I can do it by disable specific user at backend level, for example vpopmail can do it, but also with SQL can be simple. With specific query you can disable only IMAP, only POP and so on. In the password query you can insert something like this: WHERE username = '%u' AND active = '1' AND ((IF( %a=110, pop, 0 )) =1 OR (IF( %a=25, smtp , 0 )) =1 OR (IF(( %a = 143 ) AND '%l' = '127.0.0.1', webmail, 0 ) = 1 ) OR ( IF(( %a = 143 ) AND '%l' !='127.0.0.1', imap, 0 ) = 1)) Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From sven at svenhartge.de Tue Sep 4 18:40:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 17:40:17 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <20120904152007.GA20273@PC211.ikt.de> Message-ID: <0937s3tbapv8@mids.svenhartge.de> Dennis Guhl wrote: > On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: >> Hi! >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > http://wiki2.dovecot.org/Tools/Doveadm/Expunge Yes, I know about doveadm expunge. I am searching for a search_query which only matches messages with the flag \Deleted whose flag has been set 24h or longer ago. Doing something like doveadm expunge -A mailbox \* DELETED is not the solution, since it will expunge all deleted mails without looking at the time when they have been deleted. Using savedbefore does not work either, since a message may have been saved 2 years ago but only recently deleted. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Tue Sep 4 19:11:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 19:11:35 +0300 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> On 3.9.2012, at 14.31, Sven Hartge wrote: > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. Dovecot doesn't keep track of that information, so you can't. From sven at svenhartge.de Tue Sep 4 19:35:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 18:35:51 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> Message-ID: <0937vh4hfsv8@mids.svenhartge.de> Timo Sirainen wrote: > On 3.9.2012, at 14.31, Sven Hartge wrote: >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > Dovecot doesn't keep track of that information, so you can't. I suspected this. Would be nice though. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Tue Sep 4 19:37:24 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:37:24 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? Message-ID: <50462E44.2090505@Media-Brokers.com> Almost every message I'm getting through this list is duplicated, down to the same exact message-ID... Anyone else seeing this? Charles From CMarcus at Media-Brokers.com Tue Sep 4 19:40:48 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:40:48 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50462F10.90906@Media-Brokers.com> On 2012-09-04 12:37 PM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Even this one was duplicated... From philip at turmel.org Tue Sep 4 19:58:01 2012 From: philip at turmel.org (Phil Turmel) Date: Tue, 04 Sep 2012 12:58:01 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <50463319.8060309@turmel.org> On 09/04/2012 12:40 PM, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Even this one was duplicated... Not here :-) Phil From stan at hardwarefreak.com Tue Sep 4 20:02:38 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:02:38 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <5046342E.4010808@hardwarefreak.com> On 9/3/2012 11:21 PM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx What does TB activity manager say? -- Stan From stan at hardwarefreak.com Tue Sep 4 20:07:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:07:05 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50463539.9090005@hardwarefreak.com> On 9/4/2012 11:37 AM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Nope. Make any changes to Postfix or your script recently? -- Stan From florob at babelmonkeys.de Tue Sep 4 21:25:54 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 04 Sep 2012 20:25:54 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation Message-ID: <504647B2.5060804@babelmonkeys.de> Hello everyone and Timo in particular, about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present in Dovecot. I had always disliked this a bit, because it replicates a lot of code. This patch generalizes the HMAC function to take a hash_method struct as parameter, and changes existing code which uses the "old" HMAC function to use this new one. I'm not really sure this is actually a good idea, but I still felt I should provide the code in case you would want to merge it upstream. Attached is the patch as a hg export based on the revision of dovecot-2.2 current at the time of writing. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID e2f682fab829c2ef23a050f884191f57e2fb5d60 # Parent 9bc2e718392ceaa327f14b50163232b629cd54d1 lib: Generalize hmac to be hash independent diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,64 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,12 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + ctx->hash = meth; + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,23 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +struct hmac_context { + void *ctx, *ctxo; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From rob0 at gmx.co.uk Tue Sep 4 22:16:50 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 4 Sep 2012 14:16:50 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <20120904191649.GN3672@harrier.slackbuilds.org> On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rplatel at tucows.com Tue Sep 4 23:01:45 2012 From: rplatel at tucows.com (Richard Platel) Date: Tue, 4 Sep 2012 16:01:45 -0400 Subject: [Dovecot] Custom auth process in dovecot 2 Message-ID: Hi, I'm trying to upgrade from dovecot-1.1.x to 2.1.7. We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. In dovecot 1.1 we could configure this with auth external { socket connect { master { path = /var/run/dovecot/auth.sock } } } as per http://wiki.dovecot.org/MainConfig I haven't been able to figure out how to do this in 2.1.7, is it possible? From tss at iki.fi Tue Sep 4 23:05:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 23:05:55 +0300 Subject: [Dovecot] Custom auth process in dovecot 2 In-Reply-To: References: Message-ID: <99C78902-F077-4DB0-991C-39171320EF1E@iki.fi> On 4.9.2012, at 23.01, Richard Platel wrote: > I'm trying to upgrade from dovecot-1.1.x to 2.1.7. > > We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. > > In dovecot 1.1 we could configure this with > > auth external { > socket connect { > master { > path = /var/run/dovecot/auth.sock > } > } > } > > as per http://wiki.dovecot.org/MainConfig > > I haven't been able to figure out how to do this in 2.1.7, is it possible? Possibility a) Disable the regular service auth, something like: service auth { unix_listener login/login { mode = 0 } } and then just create the /var/run/dovecot/login/login socket yourself. You may need/want to do this for other auth-related sockets too. Possibility b) Tell login processes to connect to your socket instead: service imap-login { executable = imap-login newlogin } From CMarcus at Media-Brokers.com Wed Sep 5 00:02:00 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 17:02:00 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50463539.9090005@hardwarefreak.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> Message-ID: <50466C48.1020303@Media-Brokers.com> On 9/4/2012 1:07 PM, Stan Hoeppner wrote: > On 9/4/2012 11:37 AM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Nope. Make any changes to Postfix or your script recently? Nope... but, your reply made me actually take a closer look... It is only happening for emails that are filtered to folders (like my lists)... And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... I'll disable filtering on one, and see if that solves the problem (I expect it will)... Sorry for the noise... -- Best regards, Charles From h.reindl at thelounge.net Wed Sep 5 00:57:43 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 04 Sep 2012 23:57:43 +0200 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50466C48.1020303@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> Message-ID: <50467957.2010602@thelounge.net> Am 04.09.2012 23:02, schrieb Charles Marcus: > On 9/4/2012 1:07 PM, Stan Hoeppner wrote: >> On 9/4/2012 11:37 AM, Charles Marcus wrote: >>> Almost every message I'm getting through this list is duplicated, down >>> to the same exact message-ID... >>> >>> Anyone else seeing this? >> >> Nope. Make any changes to Postfix or your script recently? > > Nope... but, your reply made me actually take a closer look... > > It is only happening for emails that are filtered to folders (like my lists)... > > And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened > another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... > > I'll disable filtering on one, and see if that solves the problem (I expect it will)... better use SIEVE filters on the server for moving list-messages to folders instead the overhead of TB which is in fact copy (download/upload) followed by delete so you have no problems with concurrent clients including mobile devices -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From jtam.home at gmail.com Wed Sep 5 02:02:44 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 4 Sep 2012 16:02:44 -0700 (PDT) Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: References: Message-ID: On Tue, 4 Sep 2012, Sven Hartge wrote: > I am searching for a search_query which only matches messages with the > flag \Deleted whose flag has been set 24h or longer ago. > > Doing something like > > doveadm expunge -A mailbox \* DELETED > > is not the solution, since it will expunge all deleted mails without > looking at the time when they have been deleted. > > Using savedbefore does not work either, since a message may have been > saved 2 years ago but only recently deleted. You could work out a cron job that does a "doveadm search ...", then diff it with one generated 24 hours ago, extract the common GID/UUIDs, then do a second pass to delete them. Not elegant, but it would probably work. (I just read the man page, and there doesn't seem to be an option to pipe the search like "doveadm search ... | doveadm expunge ...". You'll have to expunge one message at a time. Ugh.) Or maybe the advice "don't worry, be happy" applies here. I find that too many "mistake mitigation" features is counterproductive as users habitually relying on them, rather than being careful. Extending your grace period to a week will lessen the probability of this race condition. Joseph Tam From kgc at corp.sonic.net Wed Sep 5 02:06:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 16:06:37 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <5046897D.1050502@corp.sonic.net> On 09/03/12 12:06, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. >> >> There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) > > Did you try stopping both and then starting them again? That clears up all the state they have. I stopped both directors last night and they were able to stay in sync after they were restarted. Could corruption of the in memory state lead to the connections being dropped? If this happens again I'll try to get a tcpdump and an strace so the bug can get squashed. -K From tss at iki.fi Wed Sep 5 03:58:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 03:58:19 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > passdb { > args = proxy=y nopassword=y > driver = static > } I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. From tss at iki.fi Wed Sep 5 04:23:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 04:23:39 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <6CB24D47-CD68-420B-A3E7-5C79DE1CD757@iki.fi> On 5.9.2012, at 3.58, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. Although the extra users should be freed from the memory after 15 minutes. Hmm. Once Dovecot supports moving existing connections from one backend server to another without the client noticing anything, the director could be simplified by using consistent hashing and when the number of backends changes, the director could start moving connections to their proper backends. During this move new connections would be handled by 1) if old backend = new backend just forward the connection there or 2) if they're different, request immediate move for that user's existing connections and wait for it to be finished before letting new connections finish. Or alternatively if the user isn't just being moved at that time, forward the connection to the old server and let it be part of the later move. The main difference here is that directors wouldn't need to keep any track of user -> backend associations. The moving period could still be a bit tricky to handle well, especially since the situation can change again while a previous move is still going on. From kgc at corp.sonic.net Wed Sep 5 07:06:51 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 21:06:51 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <5046CFDB.90809@corp.sonic.net> On 9/4/2012 5:58 PM, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. It doesn't look like there was a higher than normal number of failed logins leading up to the connection issues. I'm going to write some more stats collection tools to track state on the directors and see what comes of it. Can the director proxy validate the username via a unix pw lookup but not check the password? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From p at state-of-mind.de Wed Sep 5 09:14:10 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 08:14:10 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045DDAD.3010705@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> Message-ID: <20120905061409.GC12086@state-of-mind.de> * Robert Schetterer : > Am 04.09.2012 10:30, schrieb Robert Schetterer: > > Hi, perhaps somebody wants to this > > > > --snip > > Microsoft? Outlook? 2013 Preview > > Outlook 2013 Preview implements the IMAP LIST extension specified in > > [RFC6154] as the XLIST command. > > ---snip > > > > taken out of [MS-STANOIMAP].pdf > > > > which zip you may download here > > > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > > > > just for info, by small testing > Outlook 2013 preview imap specialuse is working with dovecot > at minimum for Trash and Sent > > i have set this > > mailbox Sent { > special_use = \Sent > auto=subscribe > } > mailbox "Sent Messages" { > special_use = \Sent > } Why did you set the \Sent folder twice? Setting it once should suffice. > the preview is only in english or spanish > so dont know if it will work with other languages then english, lets hope so Try this: mailbox "Gesendete Objekte" { special_use = \Sent auto=subscribe } AFAIK the English version should automagically map itself to that folder. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From duihi77 at gmail.com Wed Sep 5 09:45:09 2012 From: duihi77 at gmail.com (Duane Hill) Date: Wed, 5 Sep 2012 06:45:09 +0000 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <723005990.20120905064509@gmail.com> On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > * Robert Schetterer : >> Am 04.09.2012 10:30, schrieb Robert Schetterer: >> > Hi, perhaps somebody wants to this >> > >> > --snip >> > Microsoft? Outlook? 2013 Preview >> > Outlook 2013 Preview implements the IMAP LIST extension specified in >> > [RFC6154] as the XLIST command. >> > ---snip >> > >> > taken out of [MS-STANOIMAP].pdf >> > >> > which zip you may download here >> > >> > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >> > >> >> just for info, by small testing >> Outlook 2013 preview imap specialuse is working with dovecot >> at minimum for Trash and Sent >> >> i have set this >> >> mailbox Sent { >> special_use = \Sent >> auto=subscribe >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } > Why did you set the \Sent folder twice? Setting it once should suffice. FYI: Default config shows it listed twice in: conf.d/15-mailboxes.conf ... # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } ... -- If at first you don't succeed... ...so much for skydiving. From robert at schetterer.org Wed Sep 5 10:18:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:18:14 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <5046FCB6.7080700@schetterer.org> Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > Try this: > > mailbox "Gesendete Objekte" { > special_use = \Sent > auto=subscribe > } i will do when Outlook 2013 in german got released > > AFAIK the English version should automagically map itself to that folder. agree ,it should > > p at rick thunderbirds status about xlist etc can be seen here https://bugzilla.mozilla.org/show_bug.cgi?id=558659 looks like , its on the road, from pure tec side no idea if and when it will go released -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 5 10:28:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:28:53 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <723005990.20120905064509@gmail.com> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <723005990.20120905064509@gmail.com> Message-ID: <5046FF35.4070504@schetterer.org> Am 05.09.2012 08:45, schrieb Duane Hill: > On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > >> * Robert Schetterer : >>> Am 04.09.2012 10:30, schrieb Robert Schetterer: >>>> Hi, perhaps somebody wants to this >>>> >>>> --snip >>>> Microsoft? Outlook? 2013 Preview >>>> Outlook 2013 Preview implements the IMAP LIST extension specified in >>>> [RFC6154] as the XLIST command. >>>> ---snip >>>> >>>> taken out of [MS-STANOIMAP].pdf >>>> >>>> which zip you may download here >>>> >>>> http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >>>> >>> >>> just for info, by small testing >>> Outlook 2013 preview imap specialuse is working with dovecot >>> at minimum for Trash and Sent >>> >>> i have set this >>> >>> mailbox Sent { >>> special_use = \Sent >>> auto=subscribe >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } > >> Why did you set the \Sent folder twice? Setting it once should suffice. > > FYI: Default config shows it listed twice in: > > conf.d/15-mailboxes.conf > ... > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > ... > "Sent Messages" was/is used in some clients for Sent, guess its also in the example, looks like the new client versions with xlist etc changed this sometimes,too ( going directly Sent , but i didnt investigated hardly in this, test yourself), so it may not longer needed in the future, meanwhile it does not hurt somenone, since yet But i agree feature need more real world examples and explains in some wiki related to div mail clients in their different versions guess this is on the road -- Best Regards MfG Robert Schetterer From dafan.zhai at securepoint.de Wed Sep 5 11:53:37 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Wed, 05 Sep 2012 10:53:37 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder Message-ID: <50471311.2050002@securepoint.de> Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, &metadata); but Valgrind finds a memory leak when this function is called: ---------------------------------------------------snip-------------------------------------------------------------- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304== by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---------------------------------------------------snip-------------------------------------------------------------- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---------------------------------------------------snip-------------------------------------------------------------- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---------------------------------------------------snip-------------------------------------------------------------- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1 branch fixing memory leaks: http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b It would be nice if you could give me some hints how I can address this issue? Thanks in advance! Greetings, Dafan From lgb at lgb.hu Wed Sep 5 13:20:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 5 Sep 2012 12:20:26 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? Message-ID: <20120905102026.GB23439@vega.lgb.hu> Dear All, I have a bit complex question about ldap based user/passdb with some twists caused by the LDAP schema I have to use. It's complex for me at least, since it's my first time to try to setup any dovecot install which is more than a "trivial" thing eg for some unix system users. What I'd like to do: Users are stored in LDAP with the following schema: storageMailUid: contains user's uid in x at z format, and it CAN be different than mail address. This value is used to login (pop3/imap) and to get the maildir path (well, home) in the form of /mailstorage/z/x/ regardless of the value of the "mail" attribute. If @z part is missing (I mean the user supplied login, not the LDAP!), some default value (example.com) should be treated (I guess that's easy: auth_default_realm may be enough, isn't it). It is NOT possible to login via pop3/imap with mail, only with storageMailUid! This storageMailUid can specify a string which is not even an existing mail address, of course. mail: contains user's mail address, or even mail addresses (there can be more mail addresses - more mail ldap attributes - for a single ldap entry). This attribute cannot be used for pop3/imap login, neither it counts to resolve the maildir path, it only counts when a mail is received: the location of the user's maildir must be got from storageMailUid. storageMailQuota: it contains the quota value (in bytes) for the given user, which should be enforced on receiving mail, or by using IMAP (not counting the Trash folder which is a fixed size for all users and it must be handled outside of this). All user maildirs have the same fixed unix UID/GID, which is not stored in LDAP, but must be configured statically. "Home directory" of the user is also not stored in LDAP, it must be got from storageMailUid LDAP parameter. Mails are received via LMTP only. I'd like to use prefetchdb to minimalize the amount of LDAP lookups (however I can live without that). I am unsure if auth_bind is OK, or it should not be used, currently I'd like to play with auth_bind, since it worked before on other servers well. I'm totally lost with the user_attrs/pass_attrs to create this kind of configuration. In case of static userdb, it was easy to set up: passdb { args = /etc/dovecot/dovecot-ldap-passdb.conf driver = ldap } userdb { args = uid=vmail gid=vmail home=/mailstorage/%Ld/%Ln driver = static } With this, pop3/imap worked (quota was not of the scope yet, though), however lmtp is not ("passdb doesn't support lookups, can't verify user's existence" which is odd for me, as userdb does not support it in my opinion). Please, give some suggestion how to set up userdb and passdb to support this configuration which works with the described scenario. I should also set iterate_filter and iterate_attrs I guess, so some doveadm commands can work then (-A stuffs, I guess). I am also not sure if auth service should be used or not (I mean extra configuration related): since I want LMTP not LDA, I guessed I don't need it, but I am not sure. I am also confused, because on receiving a mail (via LMTP) different kind of LDAP lookup is needed: then mail must be searched, but it's storageMailUid based lookup in case of pop3 or imap login ... Is it possible to give different userdb/passdb for lmtp and pop3/imap? In theory it's even possible to have x at example.com as mail and y at example.com as storageMailUid for one user, and the opposite for the another, so lookups cannot be done together for mail and storageMailUid. The LDAP schema/rest of the system works this way, not an option to change. Any help is greatly welcomed. Thanks a lot in advance, G?bor From bangkokmaco at gmail.com Wed Sep 5 13:23:25 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 17:23:25 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <5046342E.4010808@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner wrote: > On 9/3/2012 11:21 PM, cc young wrote: > > cannot get TB to recognize either pop3/s or imap/s server > > > > can connect just fine with: > > > > openssl s_client -connect ms1.myserver.net:993 > > . login ... > > > > but trying with TB /var/log/mail.log gets: > > > > dovecot: pop3-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > dovecot: imap-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > What does TB activity manager say? > > Activity Manager is blank as an update, went to Claws email client. it had enough feedback and gave me enough ways to tweak - got everything going. went back to TB. recognized server, but no username/password. perhaps it's not letting the full email - which is the user name - through. but who knows? no feedback. From CMarcus at Media-Brokers.com Wed Sep 5 13:58:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 06:58:02 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50467957.2010602@thelounge.net> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> Message-ID: <5047303A.5060609@Media-Brokers.com> On 2012-09-04 5:57 PM, Reindl Harald wrote: > better use SIEVE filters on the server I know, it is on my ToDo list... we only just recently migrated this server to Dovecot, and I've had my plate full with other issues, which are now mostly resolved, so I'm about ready to circle back and finish up (installing SOGo, enabling sieve, etc), but I need to spin up a new Gentoo VM first, because I want to get the current server moved over to our new VMWare environment. -- Best regards, Charles From keijser at stone-it.com Wed Sep 5 14:00:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 13:00:04 +0200 Subject: [Dovecot] managesieve problem Message-ID: <1346842804.29847.13.camel@localhost> Hi, I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) and pigeonhole (2.0.9). All works well except sieve. I've already posted on the roundcube forum (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was told I should ask here since it's more likely a dovecot issue rather than roundcube. The problem is that if I want to manage sieve filters from roundcube, I get this error in maillog: Sep 5 12:55:39 emperor dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=1575, secured Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage The managesieve daemon is listening though: emperor ~ # netstat -tlpne | grep dovecot tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 0 1275042 362/dovecot tcp 0 0 0.0.0.0:2000 0.0.0.0:* LISTEN 0 1275037 362/dovecot tcp 0 0 0.0.0.0:4190 0.0.0.0:* LISTEN 0 1275035 362/dovecot tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN 0 1275044 362/dovecot tcp 0 0 :::143 :::* LISTEN 0 1275043 362/dovecot tcp 0 0 :::2000 :::* LISTEN 0 1275038 362/dovecot tcp 0 0 :::4190 :::* LISTEN 0 1275036 362/dovecot tcp 0 0 :::993 :::* LISTEN 0 1275045 362/dovecot emperor ~ # telnet localhost 4190 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus" "NOTIFY" "mailto" "SASL" "PLAIN" "STARTTLS" "VERSION" "1.0" OK "Dovecot ready." Before I paste any configuration files, I wanted to ask first if this is a common/known issue. Else, please let me know what additional information I should provide. Thanks in advance! kind regards, L?on From CMarcus at Media-Brokers.com Wed Sep 5 14:02:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 07:02:32 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: <50473148.8050905@Media-Brokers.com> On 2012-09-05 6:23 AM, cc maco young wrote: > On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >> What does TB activity manager say? > Activity Manager is blank > > as an update, went to Claws email client. it had enough feedback and gave > me enough ways to tweak - got everything going. > > went back to TB. recognized server, but no username/password. perhaps > it's not letting the full email - which is the user name - through. but > who knows? no feedback. You're looking at the wrong end. You need to be looking at your SERVER logs, not the Clients. -- Best regards, Charles From stephan at rename-it.nl Wed Sep 5 14:16:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 13:16:19 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346842804.29847.13.camel@localhost> References: <1346842804.29847.13.camel@localhost> Message-ID: <50473483.9020702@rename-it.nl> Op 9/5/2012 1:00 PM, L?on Keijser schreef: > Hi, > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > and pigeonhole (2.0.9). All works well except sieve. I've already posted > on the roundcube forum > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > told I should ask here since it's more likely a dovecot issue rather > than roundcube. [...] > Before I paste any configuration files, I wanted to ask first if this is > a common/known issue. Else, please let me know what additional > information I should provide. The issue does look familiar and most likely it is already solved. But, the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. Upgrade first. Regards, Stephan. From bangkokmaco at gmail.com Wed Sep 5 14:18:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 18:18:40 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: On Wed, Sep 5, 2012 at 6:02 PM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: > >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner >> >wrote: >> >>> What does TB activity manager say? >>> >> > Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. >> > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. > > yes, you are right. will get back. From amateo at um.es Wed Sep 5 14:40:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 05 Sep 2012 13:40:54 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) Message-ID: <50473A46.1080807@um.es> Hello, We have a dovecot servers farm with storage in NFS and indexes in local files. Under heavy load conditions we a log of messages in the form: Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. PS: I have attached my dovecot -n output -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_privileged_group = mail mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = maildir:User quota quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es service anvil { client_limit = 2051 } service auth { client_limit = 3072 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From tss at iki.fi Wed Sep 5 14:50:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 14:50:38 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50473A46.1080807@um.es> References: <50473A46.1080807@um.es> Message-ID: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> On 5.9.2012, at 14.40, Angel L. Mateo wrote: > Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) > > We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). > > So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). From zucca at systemschmiede.com Wed Sep 5 14:59:34 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Wed, 05 Sep 2012 13:59:34 +0200 Subject: [Dovecot] strange Folder in subscription list (was: Imap Ghost Folder) Message-ID: <50473EA6.3020203@systemschmiede.com> Hi list, here's the complete Problem for easy reading again. Can someone reproduce this? I have a strange "ghostfolder" when using shares/%%n/ as prefix for shared namespace. If i swith only this setting to shares/%%u/ everything is just as wanted, only then the prefix is very long, so i would prefer the name. dovecot --version 2.1.7 dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-14-pve i686 Debian 6.0.5 simfs auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%%d/%%n/Maildir mail_plugins = acl mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u prefix = shares/%%n/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } ssl_cert = took seconds) In-Reply-To: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> Message-ID: <50474093.3010306@um.es> El 05/09/12 13:50, Timo Sirainen escribi?: > On 5.9.2012, at 14.40, Angel L. Mateo wrote: > >> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >> >> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >> >> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. > > That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). > Ok, I'll try this. But... is there any way to know how a readdir would be done? From keijser at stone-it.com Wed Sep 5 15:49:39 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 14:49:39 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <50473483.9020702@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> Message-ID: <1346849379.4102.3.camel@localhost> Hi Stephan, On Wed, 2012-09-05 at 13:16 +0200, Stephan Bosch wrote: > Op 9/5/2012 1:00 PM, L?on Keijser schreef: > > Hi, > > > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > > and pigeonhole (2.0.9). All works well except sieve. I've already posted > > on the roundcube forum > > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > > told I should ask here since it's more likely a dovecot issue rather > > than roundcube. > [...] > > Before I paste any configuration files, I wanted to ask first if this is > > a common/known issue. Else, please let me know what additional > > information I should provide. > > The issue does look familiar and most likely it is already solved. But, > the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. > > Upgrade first. Upgraded to 2.0.21. Now I get this error: Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory for substitition in storage root directory (sieve_dir=~/sieve) Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage. regards, L?on From keijser at stone-it.com Wed Sep 5 16:26:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 15:26:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346849379.4102.3.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> Message-ID: <1346851564.4102.7.camel@localhost> On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: > Upgraded to 2.0.21. Now I get this error: > > > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: > sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory > for substitition in storage root directory (sieve_dir=~/sieve) > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: > Failed to open Sieve storage. I should probably mention that I have virtual domains/users, all located in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a real user on the box. regards, L?on From stephan at rename-it.nl Wed Sep 5 16:29:46 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 15:29:46 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346851564.4102.7.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> Message-ID: <504753CA.3060609@rename-it.nl> Op 9/5/2012 3:26 PM, L?on Keijser schreef: > On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: >> Upgraded to 2.0.21. Now I get this error: >> >> >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: >> sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory >> for substitition in storage root directory (sieve_dir=~/sieve) >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: >> Failed to open Sieve storage. > > I should probably mention that I have virtual domains/users, all located > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > real user on the box. This should provide all the info you need on that: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Stephan. From p at state-of-mind.de Wed Sep 5 19:22:21 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 18:22:21 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5046FCB6.7080700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <5046FCB6.7080700@schetterer.org> Message-ID: <20120905162219.GB24756@state-of-mind.de> * Robert Schetterer : > Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > > Try this: > > > > mailbox "Gesendete Objekte" { > > special_use = \Sent > > auto=subscribe > > } > > i will do when Outlook 2013 in german got released > > > > AFAIK the English version should automagically map itself to that folder. > > agree ,it should > > > > > p at rick > > thunderbirds status about xlist etc can be seen here > > https://bugzilla.mozilla.org/show_bug.cgi?id=558659 > > looks like , its on the road, from pure tec side > no idea if and when it will go released It is implemented. We had to adapt a few extra functions to deal with 64 bit stuff. Currently it is not being pushed further because of the uncertainty of TBs future. Once it will become clear how new features will be released we will spend the rest of money and time to ship the feature. For now I will not spend a single more Euro. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From CMarcus at Media-Brokers.com Thu Sep 6 00:03:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 17:03:27 -0400 Subject: [Dovecot] mail_location / mail_home In-Reply-To: <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> References: <38719C76-8672-4F85-B4EA-8802CF85BE63@4lin.net> <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> Message-ID: <5047BE1F.2000103@Media-Brokers.com> On 2012-08-31 4:15 AM, Denny Schierz wrote: > I think, it doesn't make any difference, if I have "~" or a fixed path > :-/ On the productive system (also 2.1.9) I get messages like: [...] > dovecot.lda-dupes/tmp) failed: Not a directory [...] So, I need a way, > to move ~11.000 Maildirs .... any suggestions? mmv is a godsend... http://linux.dsplabs.com.au/mmv-copy-append-link-move-multiple-files-under-linux-shell-bash-by-wildcard-patterns-p5/ -- Best regards, Charles From ben at indietorrent.org Thu Sep 6 01:20:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 05 Sep 2012 18:20:57 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot Message-ID: <5047D049.8080004@indietorrent.org> Hello, I am attempting to use the Antispam plug-in for Dovecot and am having trouble with the implementation. My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. Mail that SpamAssassin tags as SPAM is moved into the affected user's "Junk" folder (/var/vmail/example.com/user/Maildir/.Junk). I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam . After restarting Dovecot, my IMAP client (Thunderbird v15) reports the following whenever I attempt to move mail from Inbox to "Junk" directory: "[CANNOT] antispam plugin not configured" That exact phrase appears only once in a Google search ( http://dovecot-antispam.sourcearchive.com/documentation/1.1plus-p20090218.git.g28075fa/mailtrain_8c-source.html ): if (!hamaddr || !spamaddr) { mail_storage_set_error(t->box->storage, ME(NOTPOSSIBLE) "antispam plugin not configured"); return -1; } Based on the surrounding source code, which I realize may be old, it seems that Ham and Spam addresses are required, or similar. I am using the following configuration options: /etc/dovecot/dovecot.conf -------------------- ## IMAP specific settings protocol imap { mail_executable = /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_plugins = quota imap_quota antispam } # [...] plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_spam_pattern_ignorecase = spam;junk } -------------------- For what it's worth, the logging directives seem not to have any effect (and perhaps the rest of the directives do not, either, hence the problem I'm having). I tried changing the above directives to their older directive names (I also changed the order slightly, which may be irrelevant): -------------------- plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_spam_pattern_ignorecase = spam;junk } -------------------- This changed the message in my IMAP client, when attempting to move a message from my Inbox to the Junk folder, to: "[SERVERBUG] failed to send mail" Does this indicate a problem with the "/usr/bin/sa-learn-pipe.sh" script that is provided on the Wiki (cited above)? Also, it was not clear from the documentation ( http://johannes.sipsolutions.net/files/antispam.html ) whether or not the Pipe back-end requires the X-DSPAM-Signature header, and if so, to what value it should be set when using SpamAssassin. Thanks for any pointers, -Ben From jtam.home at gmail.com Thu Sep 6 01:22:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 5 Sep 2012 15:22:31 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: Message-ID: On Wed, 5 Sep 2012, Charles Marcus wrote > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. Actually, I think he did supply that and it showed imap and pop3 not getting any auth information. > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx There are other log options to verbosely log auth and ssl information, and that might help. I frequently see the above log messages when the client and the server get their security protocol (SSL/TLS) and ports crossed up (i.e. trying IMAP/SSL on port 143 or trying IMAP/TLS on port 993). Try fiddling with TB's setting here and see if anything shakes loose. You can also use openssl and simulate the server end and maybe that will shed light on the situation: (You'll have to read the man page yourself to find all the options) openssl s_server -accept 993 ... Joseph Tam From kgc at corp.sonic.net Thu Sep 6 03:17:46 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 05 Sep 2012 17:17:46 -0700 Subject: [Dovecot] dovecot stats: useful data to gather In-Reply-To: <20120603001049.GA10970@daniel.localdomain> References: <20120601205839.GG2176@state-of-mind.de> <20120603001049.GA10970@daniel.localdomain> Message-ID: <5047EBAA.8090500@corp.sonic.net> On 06/02/12 17:10, Daniel Parthey wrote: > Patrick Ben Koetter wrote: >> following our discussion on dovecot stats at the LinuxTag 2012 my team and I >> sat down and put together a list of stat items we think to be useful in daily >> dovecot usage. >> >> Besides pulling together all the data we also think it would be useful to have >> an SNMP interface to access the stats. Our offer to create and contribute a >> standalone web interface for dovecot stats stands. > > This should be done via SNMP subagent, but how could you differentiate > different dovecot instances on the same machine, different snmp ports > for the subagent, or different snmp trees? I'd suggest some additional performance metrics like min/max/avg time to authenicate, establish a proxy session and perhaps include auth failure causes counters as well. I personally wouldn't want to see this implemented as an SNMP subagent but so long as the stats would be available off a local socket directly I think everyone would be happy. -K From stan at hardwarefreak.com Thu Sep 6 05:36:02 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 05 Sep 2012 21:36:02 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: <50480C12.5040500@hardwarefreak.com> On 9/5/2012 6:02 AM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan >> Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. He'd already checked the server logs, posted some of them, and found no useful information. Checking the client log is the next logical step, whether it turns out to contain useful information or not. In a client/server application, instructing someone to only check half the logs is bad advice Charles. Useful troubleshooting information can be found in either, or both, depending on the circumstances and operation that's failing. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 6 05:50:00 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 6 Sep 2012 04:50:00 +0200 Subject: [Dovecot] Length of attachment filenames In-Reply-To: References: Message-ID: <20120906025000.GA11887@daniel.localdomain> Fuhrmann, Hauke wrote: > I have a beginners question to Dovecot: does it have a restriction on > the supported length and character encodings of filenames of > attachments? > > Can filenames be arbitrary large? The maximum filename length depends largely on the file system: http://en.wikipedia.org/wiki/Comparison_of_file_systems Regards Daniel -- https://plus.google.com/103021802792276734820 From keijser at stone-it.com Thu Sep 6 09:25:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Thu, 06 Sep 2012 08:25:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <504753CA.3060609@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> <504753CA.3060609@rename-it.nl> Message-ID: <1346912704.32331.2.camel@localhost> On Wed, 2012-09-05 at 15:29 +0200, Stephan Bosch wrote: > > I should probably mention that I have virtual domains/users, all located > > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > > real user on the box. > > This should provide all the info you need on that: > > http://wiki2.dovecot.org/VirtualUsers/Home Works like a charm now, thanks! :) regards, L?on From lists at wildgooses.com Thu Sep 6 09:27:47 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 07:27:47 +0100 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <5047303A.5060609@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> <5047303A.5060609@Media-Brokers.com> Message-ID: <50484263.5040208@wildgooses.com> On 05/09/2012 11:58, Charles Marcus wrote: > I know, it is on my ToDo list... we only just recently migrated this > server to Dovecot, and I've had my plate full with other issues, which > are now mostly resolved, so I'm about ready to circle back and finish > up (installing SOGo, enabling sieve, etc), I have recently noticed owncloud (even has an ebuild for it). Have you re-evaluated roundcube+owncloud vs SOGo for a dav calender/contacts solution? Ed From CMarcus at Media-Brokers.com Thu Sep 6 13:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:10:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5047D049.8080004@indietorrent.org> References: <5047D049.8080004@indietorrent.org> Message-ID: <50487684.8060404@Media-Brokers.com> On 2012-09-05 6:20 PM, Ben Johnson wrote: > My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. 1.2.9 is really old... you really need to upgrade to a recent/stable version. > I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam That is for version 2.x (note the wiki2)... I recommend going ahead and upgrading to the latest 2.1.9 and starting over. -- Best regards, Charles From CMarcus at Media-Brokers.com Thu Sep 6 13:17:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:17:21 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50480C12.5040500@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> <50480C12.5040500@hardwarefreak.com> Message-ID: <50487831.3060103@Media-Brokers.com> On 2012-09-05 10:36 PM, Stan Hoeppner wrote: > He'd already checked the server logs, posted some of them, and found > no useful information. Checking the client log is the next logical > step, whether it turns out to contain useful information or not. In a > client/server application, instructing someone to only check half the > logs is bad advice Charles. Useful troubleshooting information can be > found in either, or both, depending on the circumstances and operation > that's failing. That's what I get for replying in the middle of a thread without reading the whole thing... sorry... Been really busy at $dayjob, and haven't had time to read the lists I'm on for a couple of weeks now. And I didn't advise him to *only* check half the logs, I thought he was *starting out* with the client logs. Of course you're correct that once he'd determined that no auth attempts were happening, checking the client side is the next logical step. But then someone else suggested that it was just because Thunderbird was crapware, which is just plain stupid. If Thunderbird isn't AUTH'ing, it is a config (thus, user) error. -- Best regards, Charles From lgb at lgb.hu Thu Sep 6 14:06:34 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Thu, 6 Sep 2012 13:06:34 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? In-Reply-To: <20120905102026.GB23439@vega.lgb.hu> References: <20120905102026.GB23439@vega.lgb.hu> Message-ID: <20120906110634.GA12495@vega.lgb.hu> Hi Again, On Wed, Sep 05, 2012 at 12:20:26PM +0200, G?bor L?n?rt wrote: [...] > I am also confused, because on receiving a mail (via LMTP) different > kind of LDAP lookup is needed: then mail must be searched, but it's > storageMailUid based lookup in case of pop3 or imap login ... Is it > possible to give different userdb/passdb for lmtp and pop3/imap? Ok, after some "serious" google usage, I found a message in the mail list archive, that it helps to put userdb/passdb section inside the protocol specific part of the configuration. So I have something now like this: protocol pop3 { [...] } protocol imap { [...] } protocol lmtp { [...] passdb { [...] } userdb { [...] } } passdb { [...] } userdb { [...] } I have the idea, that in this way, lmtp should use dbs specified in the lmtp specific protocol settings, any other stuffs will use the settings at the "root level" of the configuration (this also includes iteration specific filter). However eg if I try to deliver a mail through lmtp in this way to a non-existing mail user, I can see in the logs, that lmtp after trying the specific dbs, it will also try the "global" ones, which is not good for me, as it can cause mis-deliveries instead of rejecting (as I have "some at thing" formatted %u for both of uid and mail but often they are not the same for the same user). I would be able to put db specifications into pop3 and imap (so not "global" configuration for them) but I guess in this case other services may (?) fail, like doveadm stuffs with CLI swtich -A (or am I wrong here?). Also it's not as nice, since then I must duplicate the same db specifications in both of imap and pop3 protocol specification parts of the configuration even they are the very same. Is there any idea to fix this little problem and/or a suggestion to solve my problem in a more elegant way than I tried to do? Thanks a lot in advance. - G?bor From dafan.zhai at securepoint.de Thu Sep 6 19:19:27 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Thu, 06 Sep 2012 18:19:27 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <50471311.2050002@securepoint.de> References: <50471311.2050002@securepoint.de> Message-ID: <5048CD0F.9030809@securepoint.de> Hi I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Dafan On 09/05/2012 10:53 AM, Dafan Zhai wrote: > Hi everyone, > > I am writing a dovecot statistic plugin, which calls the > 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as > the 2nd parameter. > > enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; > struct mailbox_metadata metadata; > mailbox_get_metadata(mailbox, metadata_items, &metadata); > > but Valgrind finds a memory leak when this function is called: > ---------------------------------------------------snip-------------------------------------------------------------- > > ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 > of 76 > ==10304== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) > ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) > ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) > ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) > ==10304== by 0x4067102: maildir_mailbox_get_metadata > (maildir-storage.c:486) > ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) > ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) > ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) > ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) > ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) > ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) > ---------------------------------------------------snip-------------------------------------------------------------- > > > I have looked into the source code, and found that from the > 'mail_search_build_init' function call in > lib-storage/index/index-status.c:200 a pool is created, but the pool is > not freed in the mailbox_search_deinit function call in > lib-storage/index/index-status.c:218. > This may be the reason of the memory leak. But I do not know how to free > the pool. > > It seems to me that doveadm is having the same memory leak because it is > using 'mail_search_build_init' too. > ---------------------------------------------------snip-------------------------------------------------------------- > > # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize > INBOX > ==10457== Memcheck, a memory error detector > ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. > ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info > ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX > ==10457== > INBOX vsize=41643319 > ==10457== > ==10457== HEAP SUMMARY: > ==10457== in use at exit: 4,356 bytes in 3 blocks > ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated > ==10457== > ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 > ==10457== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x41AF16C: pool_alloconly_create (in > /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x40DC07D: mail_search_build_init (in > /usr/lib/dovecot/libdovecot-storage.so.0.0.0) > ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in > /usr/bin/doveadm) > ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) > ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) > ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) > ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) > ==10457== by 0x805B863: main (in /usr/bin/doveadm) > ==10457== > ==10457== LEAK SUMMARY: > ==10457== definitely lost: 4,096 bytes in 1 blocks > ==10457== indirectly lost: 0 bytes in 0 blocks > ==10457== possibly lost: 0 bytes in 0 blocks > ==10457== still reachable: 260 bytes in 2 blocks > ==10457== suppressed: 0 bytes in 0 blocks > ==10457== Reachable blocks (those to which a pointer was found) are not > shown. > ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes > ==10457== > ==10457== For counts of detected and suppressed errors, rerun with: -v > ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) > ---------------------------------------------------snip-------------------------------------------------------------- > > > Dovecot version: 2.1.9 > OS: Linux 3.0.30-dist i686 > maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX > > I have applied the following patches from the dovecot 2.1 branch fixing > memory leaks: > http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 > http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b > > > It would be nice if you could give me some hints how I can address this > issue? > > Thanks in advance! > > Greetings, > Dafan -- Follow us on Facebook: Follow us on Twitter: --------------------------------------------------------------------- Securepoint GmbH | Entwicklung Dafan Zhai Salzstr. 1 D-21335 Lueneburg http://www.securepoint.de Tele: ++49 4131 2401-0 Fax: ++49 4131 2401-50 Lueneburg HRB 1776 --------------------------------------------------------------------- CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. GEHEIMHALTUNGSPFLICHT : Dieses E-Mail und alle damit verbundenen Anlagen sind vertraulich und d?rfen nur bestimmten Personen zug?nglich gemacht werden. Sofern Sie nicht zu den angegebenen Empf?ngern geh?ren, benachrichtigen Sie bitte unverz?glich den Absender. Der Inhalt darf weder an Dritte weitergegeben noch zu anderen Zwecken verwendet werden. Die Informationen d?rfen auch nicht auf einem Datentr?ger gespeichert oder auf einen Datentr?ger kopiert werden. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1.9-lib-storage-virtual_size_add_new-memory-leak.patch Type: text/x-patch Size: 873 bytes Desc: not available URL: From ben at indietorrent.org Thu Sep 6 20:56:58 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 13:56:58 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <50487684.8060404@Media-Brokers.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> Message-ID: <5048E3EA.80009@indietorrent.org> On 9/6/2012 6:10 AM, Charles Marcus wrote: > On 2012-09-05 6:20 PM, Ben Johnson wrote: > >> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu > 10.04. > > 1.2.9 is really old... you really need to upgrade to a recent/stable > version. Thanks, Charles. I do see your point. One of the challenges we face in this regard is that we're using a Long-Term-Support version of Ubuntu (10.04) and 1.2.9 is the latest package in the OS's repository. That said, we could upgrade manually, but this is a production server on which downtime must be minimized, and we all know how unexpected issues arise during installation (even when the procedure is tested in a closely equivalent development environment). >> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam > > That is for version 2.x (note the wiki2)... Noted; thank you. > I recommend going ahead and upgrading to the latest 2.1.9 and starting > over. > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle with this further. Thanks again, -Ben From lists at wildgooses.com Thu Sep 6 21:19:03 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 19:19:03 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <5048E917.6030004@wildgooses.com> On 06/09/2012 18:56, Ben Johnson wrote: > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). I personally use (lightweight) virtualisation on any new machine, I really don't see any reason why NOT to. I would typically also setup my mounts such that the operating system is separate from "the data". This makes it easy to upgrade the OS/services, but without touching the data (test before/after on the same data for example) So in my situation I would boot a fairly small (gentoo in my case) virtual environment that runs only dovecot + postfix, it mounts the mail spools separately - I say "boot", but because I'm using linux-vservers, it's really a fancy chroot, and so the instance will start in 2-3 seconds (restarts are similarly near instant). I would upgrade by cloning this installation, upgrading it, testing it to bits, and then to make it live basically you swap this "machine" for the live machine. There are various ways it could be made near seamless, but in my situation I can bear a couple of seconds whilst I literally restart the "machine" Similarly I segregate all my services into a dozen or so "virtual machines", so DNS has it's own "machine" and so does logging, databases, almost every webservice gets its own virtual environment, etc. You could use a full blown vmware/kvm/etc if that floats your boat better, but the point remains it's so trivial to install, makes upgrades to trivial and massively decreases your downtime risk that it's very hard to find a reason NOT to do it... I haven't tried too hard to keep my instances tiny, so each is probably around 400-600MB in my case. However, if it were important this could easily be reduced to 10-100s MB each using various hardlink features. As you can see it's easy to snapshot a whole machine to manage upgrades/backups, etc This is more about infrastructure, but I honestly can't get over how many people are sitting on their hands shackled by "I'm on Debian xxx and I can't install any software newer than 5 years old"... It's so easy to escape from that trap...!! Good luck Ed W From ben at indietorrent.org Thu Sep 6 22:38:07 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 15:38:07 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E917.6030004@wildgooses.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <5048E917.6030004@wildgooses.com> Message-ID: <5048FB9F.7030104@indietorrent.org> On 9/6/2012 2:19 PM, Ed W wrote: > On 06/09/2012 18:56, Ben Johnson wrote: >> >> On 9/6/2012 6:10 AM, Charles Marcus wrote: >>> On 2012-09-05 6:20 PM, Ben Johnson wrote: >>> >>>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >>> 10.04. >>> >>> 1.2.9 is really old... you really need to upgrade to a recent/stable >>> version. >> Thanks, Charles. I do see your point. One of the challenges we face in >> this regard is that we're using a Long-Term-Support version of Ubuntu >> (10.04) and 1.2.9 is the latest package in the OS's repository. >> >> That said, we could upgrade manually, but this is a production server on >> which downtime must be minimized, and we all know how unexpected issues >> arise during installation (even when the procedure is tested in a >> closely equivalent development environment). > > I personally use (lightweight) virtualisation on any new machine, I > really don't see any reason why NOT to. I would typically also setup my > mounts such that the operating system is separate from "the data". This > makes it easy to upgrade the OS/services, but without touching the data > (test before/after on the same data for example) Thanks for your valuable insights, Ed. That seems like a worthwhile approach. > So in my situation I would boot a fairly small (gentoo in my case) > virtual environment that runs only dovecot + postfix, it mounts the mail > spools separately - I say "boot", but because I'm using linux-vservers, > it's really a fancy chroot, and so the instance will start in 2-3 > seconds (restarts are similarly near instant). I would upgrade by > cloning this installation, upgrading it, testing it to bits, and then to > make it live basically you swap this "machine" for the live machine. > There are various ways it could be made near seamless, but in my > situation I can bear a couple of seconds whilst I literally restart the > "machine" > > Similarly I segregate all my services into a dozen or so "virtual > machines", so DNS has it's own "machine" and so does logging, databases, > almost every webservice gets its own virtual environment, etc. You could > use a full blown vmware/kvm/etc if that floats your boat better, but the > point remains it's so trivial to install, makes upgrades to trivial and > massively decreases your downtime risk that it's very hard to find a > reason NOT to do it... While I'm with you here, and I understand the theory (and practice, to some extent), doesn't all of this require a true, physical machine? We can't justify the expense associated with a physical machine in a hosted environment, so we're left with so-called VPSs. My understanding is that OpenVZ cannot be installed on a VPS (for seemingly obvious reasons -- namely, that the VPS is itself an OpenVZ container). > I haven't tried too hard to keep my instances tiny, so each is probably > around 400-600MB in my case. However, if it were important this could > easily be reduced to 10-100s MB each using various hardlink features. > As you can see it's easy to snapshot a whole machine to manage > upgrades/backups, etc > > > This is more about infrastructure, but I honestly can't get over how > many people are sitting on their hands shackled by "I'm on Debian xxx > and I can't install any software newer than 5 years old"... It's so easy > to escape from that trap...!! Perhaps easy, but not necessarily inexpensive. ;-) Thanks again for sharing the details of your strategy; I'll bear all of this in mind moving forward. > Good luck > > Ed W > > -Ben From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 01:12:58 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 00:12:58 +0200 Subject: [Dovecot] dovecotadm error In-Reply-To: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> References: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> Message-ID: <20120906221258.GA9219@daniel.localdomain> Hi Michael, Michael wrote: > doveadm expunge -A mailbox Junk savedbefore 28d > doveadm(root): Error: User listing returned failure > > The mail log file says: > dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table > 'system.users' doesn't exist (using built-in default iterate_query: > SELECT username, domain FROM users) > > dovecot-sql.conf: > iterate_query = SELECT username AS user FROM accounts Some examples in the wiki refer to "username", you could try this: iterate_query = SELECT username FROM accounts > though I have had to comment the following from dovecot.conf: > #iteratedb { > # args = /usr/local/etc/dovecot/dovecot-sql.conf > # driver = sql > #} > because it errors on dovecot start up. Your primary userdb refers to dovecot-sql.lda.conf, not dovecot-sql.conf. What does /usr/local/etc/dovecot/dovecot-sql.lda.conf look like and is there an iterate_query inside this file? Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at knoxvillechristian.org Fri Sep 7 01:50:19 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 06 Sep 2012 18:50:19 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <504928AB.7040702@knoxvillechristian.org> Turn on more debugging with these in your configuration: auth_verbose = yes auth_debug = yes auth_debug_passwords = yes mail_debug = yes verbose_ssl = yes and then check the server logs after trying to login. Also, I'm curious why you masked your PRIVATE ip address and not your public one. Bill On 9/4/2012 12:21 AM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 02:56:37 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 01:56:37 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 Message-ID: <20120906235637.GA11324@daniel.localdomain> Hi, I'm getting segfaults and unexpected disconnects from managesieve server, when the Thunderbird SIEVE extension tries to validate SIEVE scripts agains Pidgeonhole in Dovecot 2.1.8. The extension says: "Server terminated unexpectedly the connection, click on reconnect to try again." It can be reproduced by simply calling CHECKSCRIPT followed by a string: mail01:~# telnet 127.0.0.1 19200 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. "IMPLEMENTATION" "Sieve" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave" "NOTIFY" "mailto" "SASL" "PLAIN" "VERSION" "1.0" OK "Mailbox" AUTHENTICATE "PLAIN" "AGRwYXJ0aGV5QGV4YW1wbGUub3JnAGRwYXJ0aGV5" OK "Logged in." CHECKSCRIPT {6+} abcdef Connection closed by foreign host. The server interrupts the connection and the logs show the following: # dovecot.log Sep 7 01:40:46 dovecot: mailbox: mail: managesieve(dparthey at example.org): Fatal: master: service(managesieve): child 31356 killed with signal 11 (core dumped) # kern.log Sep 7 01:40:46 kernel: [1417105.954609] managesieve[31356]: segfault at 0 ip 00007f1c415c4876 sp 00007fffb3731f88 error 4 in libc-2.11.1.so[7f1c41543000+17a000] Here is the backtrace: mail01:~# gdb /usr/lib/dovecot/managesieve /var/tmp/core.managesieve.31356 Core was generated by `dovecot-mailbox/managesieve'. Program terminated with signal 11, Segmentation fault. #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 (gdb) bt full #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 No symbol table info available. #1 0x000000000040b1c3 in sieve_storage_save_will_activate (ctx=0x257f1a0) at sieve-storage-save.c:328 _data_stack_cur_id = 4 scriptname = 0x25400ae "default.sieve" ret = #2 0x00000000004065b0 in cmd_putscript_finish_parsing (cmd=0x2575cb8) at cmd-putscript.c:206 ehandler = cpflags = sbin = errors = _data_stack_cur_id = 0 script = client = 0x2575c30 ctx = 0x2578180 args = 0x254bc68 ret = #3 0x0000000000406838 in cmd_putscript_continue_script (cmd=0x2575cb8) at cmd-putscript.c:423 all_written = client = 0x2575c30 ctx = 0x2578180 size = 39279968 #4 0x0000000000406caf in client_input_putscript (context=0x2575c30) at cmd-putscript.c:84 cmd = 0x2575cb8 __FUNCTION__ = "client_input_putscript" #5 0x00007f1c41d1d3e6 in io_loop_call_io (io=0x2575fd0) at ioloop.c:379 ioloop = 0x2548680 t_id = 2 #6 0x00007f1c41d1e46f in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 ctx = 0x25489f0 event = 0x2548a60 list = 0x2576020 io = 0x0 tv = {tv_sec = 18, tv_usec = 988469} msecs = ret = i = 0 call = false #7 0x00007f1c41d1d388 in io_loop_run (ioloop=0x2548680) at ioloop.c:398 No locals. #8 0x00007f1c41d09653 in master_service_run (service=0x2548530, callback=0x25400ae) at master-service.c:543 No locals. #9 0x00000000004096ce in main (argc=1, argv=0x2548370) at main.c:308 set_roots = {0x610d60, 0x0} login_set = {auth_socket_path = 0x2540088 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x409810 , failure_callback = 0x409290 } service_flags = storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = xxx imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } From stephan at rename-it.nl Fri Sep 7 03:14:50 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 07 Sep 2012 02:14:50 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 In-Reply-To: <20120906235637.GA11324@daniel.localdomain> References: <20120906235637.GA11324@daniel.localdomain> Message-ID: <50493C7A.1070602@rename-it.nl> On 9/7/2012 1:56 AM, Daniel Parthey wrote: > Hi, > > I'm getting segfaults and unexpected disconnects from managesieve server, > when the Thunderbird SIEVE extension tries to validate SIEVE scripts > agains Pidgeonhole in Dovecot 2.1.8. > > The extension says: > "Server terminated unexpectedly the connection, click on reconnect to try again." > > It can be reproduced by simply calling CHECKSCRIPT followed by a string: This is a known and fixed problem: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/6ceeb6421231 I should make a release soon. Regards, Stephan. From Nico.Weinreich at drefa-msg.de Fri Sep 7 10:30:46 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Fri, 7 Sep 2012 09:30:46 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars Message-ID: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Hi! I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht" => Roundcube shows that messages folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht123" => Roundcube shows error that no messages where found folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung" => Roundcube shows that messages folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung123" => Roundcube shows error saying: "Connection to storage server failed" Looking into /var/log/mail.log shows: Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4478a) [0 x7f065ba8e78a] -> /usr/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f065ba8e7d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f065ba655ef ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a3cb) [0x7f065bd6f3cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a8aa) [0x7f065bd6f8 aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x78641) [0x7f065bd4d641] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_fore ach+0x3c) [0x7f065bd4d7bc] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header+0x29) [0x7f065ba78eb9] -> /usr/lib/dovecot/libdovecot-st orage.so.0(+0x9a0cf) [0x7f065bd6f0cf] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9b160) [0x7f065bd70160] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_storage_search_next_nonblock+0x48) [0x7f065bd706e8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonbl ock+0x27) [0x7f065bd51a07] -> dovecot/imap() [0x41668f] -> dovecot/imap(imap_search_start+0xd8) [0x416ad8] -> dovecot/imap(cmd_sort+0x1e6) [ 0x40e636] -> dovecot/imap(command_exec+0x3d) [0x41157d] -> dovecot/imap() [0x4104fe] -> dovecot/imap(client_handle_input+0x135) [0x410805] - > dovecot/imap(client_input+0x5f) [0x41112f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f065ba9a926] -> /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f065ba9b95f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f065ba9a8c8] -> /usr/li b/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f065ba87053] -> dovecot/imap(main+0x2a4) [0x419354] -> /lib/libc.so.6(__libc_start_ma in+0xfd) [0x7f065b706c8d] -> dovecot/imap() [0x408e79] Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? Regards From ml at cibest.com Fri Sep 7 11:46:42 2012 From: ml at cibest.com (Cibest ML) Date: Fri, 07 Sep 2012 10:46:42 +0200 Subject: [Dovecot] Wrong quota calculation Message-ID: <5049B472.8060407@cibest.com> Hi, We've a strange quota behavior on our dovecot servers. It seems that it's somehow doubling the real used storage (or could be something like twice the inbox size + size of other folders, hard to know for sure). It was initial setup with the maildir++ system. Now it's been configured with dict and mysql table but the result is the same. Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used As example, let's use mailbox at domain.com. If I check in the quota table, I'll find : - size : 697151354 - number of files : 1510 Now if I check the Maildir folder on the filesystem, I'll find : - size : 347960000 - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, not 100% accurate) Another mail box Reported by dovecot : 503446986 1081 Filesystem : 248592000 598 Does anyone has an idea of what could be possibly wrong. We've checked & double-checked everything and nothing seems to be wrong (no dsync used). -- Regards S?bastien From amateo at um.es Fri Sep 7 13:11:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:11:45 +0200 Subject: [Dovecot] best way to activate quota Message-ID: <5049C861.2030904@um.es> Hello, I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). What do you think is the best to activate it? I have think that I could use doveadm commands to calculate an initial quota (on low load periods) and then activate quota, hoping that although quota is not correct, it would be used. Should this work? From robert at schetterer.org Fri Sep 7 13:28:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 07 Sep 2012 12:28:45 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: <5049CC5D.7070005@schetterer.org> Am 07.09.2012 12:11, schrieb Angel L. Mateo: > Hello, > > I'm planning to activate quota control in dovecot, with maildir > quota backend. I have about 70k users in my system directed to 4 backend > servers (with a director to ensure that a user is always directed to the > same server). sounds good, first look > > I have tried to activate it in one of my nodes. The problem is that > load of it has increase a lot, as much as the system was unusable > (maildir is in nfs storage, with indexes in local disks). that might not be the optimal storage solution > > What do you think is the best to activate it? yes ,until its better investigated, cause your systems shouldnt get unusable > > I have think that I could use doveadm commands to calculate an > initial quota (on low load periods) and then activate quota, hoping that > although quota is not correct, it would be used. Should this work? > > with 70 k users , you should be able to hire professional help from timo and/or other dovecot specialists however you ever will have to post your config/logs/general-setup-big-picture if asking for help -- Best Regards MfG Robert Schetterer From amateo at um.es Fri Sep 7 13:41:40 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:41:40 +0200 Subject: [Dovecot] Removing specific entry in user/auth cache In-Reply-To: <1341388872.2689.3.camel@innu> References: <1340865829.25551.64.camel@innu> <42916718-6B7E-4632-8C61-AA8FE64D850E@iki.fi> <1341388872.2689.3.camel@innu> Message-ID: <5049CF64.8090908@um.es> El 04/07/12 10:01, Timo Sirainen escribi?: > On Fri, 2012-06-29 at 05:01 +0300, Timo Sirainen wrote: >> and for v2.1 a bit kludgy way: >> >> doveadm auth [] >> doveadm auth cache flush [] > > Done: http://hg.dovecot.org/dovecot-2.1/rev/007bf0047ab0 > http://hg.dovecot.org/dovecot-2.1/rev/1093c74f54af > Hello, After some time I have updated my system to 2.1.9 which includes this patch but I have doubts it is working. I have changed an attribute for one of my users (his home directory) so I run: root at myotis33:~# doveadm auth cache flush 2 cache entries flushed but, then, when I run "doveadm user " I've got the old information, not the updated one. I had to reload dovecot to get the information correctly reloaded. From hmoreno at gmv.com Fri Sep 7 13:43:19 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Fri, 7 Sep 2012 10:43:19 +0000 Subject: [Dovecot] Quota less mailbox with mdbox Message-ID: Good morning everyone, I have a little doubt about setting a quotaless mailbox with mdbox. Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. The migration is simple and easy, but there are some configuration settings we cannot set to work properly. In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. This is working like a charm. Dovecot.conf: ... namespace { separator = . prefix = inbox = yes type = private } namespace { inbox = no location = prefix = INBOX. separator = . hidden = yes type = private list = no } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } ... plugin/quota = maildir:_backup:ns= ... Dovecot-ldap.conf: ... pass_filter = (&(objectClass=qmailUser)(uid=%u)(accountActive=TRUE)) pass_attrs = uid=user,userPassword=password, mailQuotaSize=quota_rule=*:bytes=%$ user_attrs = mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ user_filter = (&(objectClass=qmailUser)(|(mailAlternateAddress=%u)(mail=%u)(uid=%u))(accountActive=TRUE)) ... When migrated to mdbox and changing the configuration, we don't know how to get it work. We have also tried to follow the instructions from the wiki of dovecot2 without success. Could anyone give us a clue please? If additional information is needed, please let me know. Thank you very much in advanced. Kind regards. ________________________________ [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\logo_gmv.gif] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_blog.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_facebook.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_twitter.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_youtube.gif] P Please consider the environment before printing this e-mail. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From kiwi at oav.net Fri Sep 7 14:30:42 2012 From: kiwi at oav.net (Xavier Beaudouin) Date: Fri, 7 Sep 2012 13:30:42 +0200 Subject: [Dovecot] dsync mirror and sieve ? Message-ID: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Hi there, I am currently doing some extensive test with dsync, but it appear that : dsync -u maildir:/path/to/new/Maildir doesn't seems to sync sieves rules. Is there a good way to sync sieves as well ? :p Regards, Xavier From mhinch at hinchfamily.com Fri Sep 7 14:39:24 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Fri, 07 Sep 2012 06:39:24 -0500 Subject: [Dovecot] dovecot/log not running Message-ID: <5049DCEC.5050008@hinchfamily.com> I'm running 2.1.7 as downloaded/installed by Debian (and then appropriately setup for my config, ...). It almost works. I can send/receive email from thunderbird, squirrelmail, ... However there are two anomalies: 1. dovecot/log is not running. After restarting dovecot (and waiting 30 seconds), I have this in the log files: Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 After 30 seconds, a "ps -ef | grep dovecot" yields: dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil root 25148 20223 0 18:55 ? 00:00:00 dovecot/config dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). A "doveadm log find" dumps this in the log: Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) However, further debugging that I try to do (see next problem) does not result in ANY further input to the log files, even though I have all known logging turned on. So I *think* that logging is not actually running. If I can get that started, then I'll take a crack at the next problem myself ... 2. However, as long as I have your attention, my next problem is that it seems like dovecot/imap-login hangs(?). After a few hours, I have something like: # ps -ef | grep dovecot dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth root 13277 31186 0 17:47 ? 00:00:00 dovecot/config dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil with exactly 100 imap-login processes. I'm the only meaningful user on the machine (unless somebody is imap DoS-ing me), so this does not make much sense. Obviously thunderbird is polling the machine and each time is leaving around a imap-log process. As far as I can tell, thunderbird successfully logs in (i.e., I get mail) though I can't be sure whether these are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? After a while of course, dovecot will not start any new imap-login processes so Thunderbird cannot get any more mail until I restart dovecot. Then everything is OK for another 12 hours or so. Ideas on either problem? -Mark From CMarcus at Media-Brokers.com Fri Sep 7 14:59:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 07 Sep 2012 07:59:44 -0400 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <5049E1B0.2030602@Media-Brokers.com> On 2012-09-07 6:43 AM, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. If you're going to migrate to mdbox, you'd be much better off if you upgraded to the latest 2.1.x... Mdbox is much better supported in the current stable builds. -- Best regards, Charles From trashcan at odo.in-berlin.de Fri Sep 7 15:16:14 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 07 Sep 2012 14:16:14 +0200 Subject: [Dovecot] =?utf-8?q?dsync_mirror_and_sieve_=3F?= In-Reply-To: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> References: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Message-ID: <0ae24afcca736872e3795e930612cb67@mx1.enfer-du-nord.net> Hi -- Xavier Beaudouin wrote: > I am currently doing some extensive test with dsync, but it appear > that : > > dsync -u maildir:/path/to/new/Maildir > > doesn't seems to sync sieves rules. > > Is there a good way to sync sieves as well ? :p I'm using Unison to sync my user's dedicated sievedirs. Regards, Michael From tth at rfa.cz Fri Sep 7 18:34:34 2012 From: tth at rfa.cz (TeTeHacko) Date: Fri, 07 Sep 2012 17:34:34 +0200 Subject: [Dovecot] pop3 migration In-Reply-To: <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> References: <503632C7.2070008@rfa.cz> <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> Message-ID: <504A140A.6070000@rfa.cz> Probably on server side ... the problem was only on big mailboxes which is used by IMAP, so there is no problem and we have already migrated all to dovecot. ;) Thanks for your reply. On 31.8.2012 15:46, Timo Sirainen wrote: > On 23.8.2012, at 16.40, TeTeHacko wrote: > >> I have problems with pop3 migration, the server disconect the dovesync when >> fetching mails after 5 minutes with this error in server log: >> >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6548 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 47522 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6549 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 355289 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6550 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 9638 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6551 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 2188 octets >> xx.xx.xx.xx [28F4] 11:59:05 >>> -ERR Command unrecognized: "" >> xx.xx.xx.xx [28F4] 11:59:05 *** 6551 477693894 00:12:55 ERROR >> xx.xx.xx.xx [28F4] 11:59:05 Disconnected > > Looks like a bug. But in Dovecot POP3 client or the POP3 server? Apparently either Dovecot sends an empty line after RETR or the server thinks Dovecot sends it. Can you get the actual TCP traffic logs? and/or set pop3c_rawlog_dir and see if it's there. > From trashcan at odo.in-berlin.de Fri Sep 7 22:22:48 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 7 Sep 2012 21:22:48 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? Message-ID: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Hi -- Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I just want to be sure that I haven't overseen such a tool/functionality in the documentation. Did I or didn't I? Thanks and regards, Michael From Ralf.Hildebrandt at charite.de Fri Sep 7 22:30:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 7 Sep 2012 21:30:32 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907193032.GD11025@charite.de> * Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. > > Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I think this can be done with post-login scripting, where a script checks if the user is allowed to login. http://wiki.dovecot.org/PostLoginScripting -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From silent_wumpus at yahoo.com Fri Sep 7 22:32:38 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Fri, 7 Sep 2012 12:32:38 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method Message-ID: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From rostetter at mail.utexas.edu Fri Sep 7 23:10:30 2012 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Fri, 07 Sep 2012 15:10:30 -0500 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907151030.44362ukqt9598wqo@mail.ph.utexas.edu> Quoting Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot > (accepted by postfix), temporarily. Now, I would like to know if > there is a way to make a "doveadm kick -A" a permanent disconnect > until one would allow reconnects, again? This should be done > *without* shutting down dovecot or *extensive* re-configuration. A > "doveadm block -u -m 'sorry: running maintenance, please, come > back later'" alike tool/funtionality would be ideal. What I've been doing so far, is I configure dovecot 2.1 with a deny-hosts file (in auth-deny.conf.ext) and then I can append a name to that file to block them, and remove them to allow them back in. Since this file is a simple text file with one username/address per line, it is very easy to manage. No idea if that would work for you, since your usage is for a different reason than mine. -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 23:46:50 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 22:46:50 +0200 Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <1346732123552-37388.post@n4.nabble.com> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> <1346732123552-37388.post@n4.nabble.com> Message-ID: <20120907204650.GA9602@daniel.localdomain> bmalik wrote: > when i login to the server using pop3s, i see in the dovecot log; > > @4000000050457dd9179c653c pop3-login: Info: Login: user=, > method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, > session= > > if i add my mail address into the users-with-forced-encryption file, > i would not reach my account. > > i get an error; > @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script > denied access to user test at byserver.net > whereas i connect to the account via pop3s > if i remove my mail address from the users-with-forced-encryption > there is no problem. > > my postlogin_pop3.sh; > #!/usr/local/bin/bash > if [ "$SECURED" == "secure" ] > then > :# handle secure connections here > else > :# handle insecure connections here > if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption > then > printf "* NO [ALERT] You are not allowed to connect without > encryption\r\n" > exit 0 > fi > fi > exec "$@" > ########### > there is also no problem related to SSL certificate > what can be the issue ? Can you log the value of "$SECURED" to a logfile (using "logger" tool), and log a message in each of the conditional branches of your script, so we can see which branch gets called? Which error message do you get when you manually try to log into POP3S? openssl s_client -connect ip:port USER username PASS password QUIT Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 00:08:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 23:08:33 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1346745602879-37394.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> Message-ID: <20120907210833.GA10535@daniel.localdomain> bmalik wrote: > I upgraded dovecot from 1.1 to 2.1.8 > > pop3/imap works but i have a problem. > > i use pop3 protocol on outlook 2010 for my account. and i set keep my mails > on the server for 5 days or until i delete them. > These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. > if i even delete a mail from outlook the server doesn't delete or old mails > are not deleted. Dovecot does not remove expunged mails from mdbox automatically. You can run doveadm purge via cron to remove unreferenced messages: http://wiki2.dovecot.org/Tools/Doveadm/Purge Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 02:38:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 8 Sep 2012 01:38:44 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5049B472.8060407@cibest.com> References: <5049B472.8060407@cibest.com> Message-ID: <20120907233844.GA12996@daniel.localdomain> Cibest ML wrote: > We've a strange quota behavior on our dovecot servers. > It seems that it's somehow doubling the real used storage (or could > be something like twice the inbox size + size of other folders, hard > to know for sure). > It was initial setup with the maildir++ system. Now it's been > configured with dict and mysql table but the result is the same. > > Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used > As example, let's use mailbox at domain.com. > If I check in the quota table, I'll find : > - size : 697151354 > - number of files : 1510 > > Now if I check the Maildir folder on the filesystem, I'll find : > - size : 347960000 > - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, > not 100% accurate) > > Another mail box > Reported by dovecot : 503446986 1081 > Filesystem : 248592000 598 > > Does anyone has an idea of what could be possibly wrong. We've > checked & double-checked everything and nothing seems to be wrong > (no dsync used). What happens after quota resync? Does the quota change? What happens to the "doveadm quota get" output after LMTP/LDA delivery of a mail with a fixed-size, let's say one mail with 1 MB? How does your setup look like? Mailbox+Director Proxy or only Mailboxes? What is your doveconf -n output of mailbox (and possibly director)? Regards Daniel -- https://plus.google.com/103021802792276734820 From dieterknopf at googlemail.com Sat Sep 8 08:11:34 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 8 Sep 2012 07:11:34 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering Message-ID: Hello, is there any update about the sieve command line tool? I really need some way to refilter my mails without loosings flags and without moving all messages in a different folder. This is possible with procmail via direct access to the file system without the need of getmail. Thanks From robert at schetterer.org Sat Sep 8 09:14:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 08:14:27 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AE243.60207@schetterer.org> Am 08.09.2012 07:11, schrieb Dieter Knopf: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. > > This is possible with procmail via direct access to the file system > without the need > of getmail. > > Thanks > look http://wiki2.dovecot.org/Pigeonhole/Sieve there is some special sieve stuff to filter against external progs i.e. procmail -- Best Regards MfG Robert Schetterer From bmalik at ihlas.net.tr Sat Sep 8 10:25:58 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 00:25:58 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <20120907210833.GA10535@daniel.localdomain> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> Message-ID: <1347089158969-37479.post@n4.nabble.com> But i use Maildir ? is there a solution ? Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i got duplicate mails which i have kept for some days on the server. how can i prevent this problem? -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html Sent from the Dovecot mailing list archive at Nabble.com. From stephan at rename-it.nl Sat Sep 8 10:39:06 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 08 Sep 2012 09:39:06 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AF61A.9090906@rename-it.nl> On 9/8/2012 7:11 AM, Dieter Knopf wrote: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. For Pigeonhole v0.3 it is part of the normal release as the sieve-filter command. I still haven't received much feedback on it though, so make sure your backups are in order before you give it a try and please report back your findings. Regards, Stephan. From robert at schetterer.org Sat Sep 8 16:10:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:10:54 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347089158969-37479.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> Message-ID: <504B43DE.8050008@schetterer.org> Am 08.09.2012 09:25, schrieb bmalik: > But i use Maildir ? > > is there a solution ? > > Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i > got duplicate mails which i have kept for some days on the server. > how can i prevent this problem? > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html > Sent from the Dovecot mailing list archive at Nabble.com. > pop3 feature leave mails on the servers works i.e with thunderbird with a local list in Tb what is allready download for compare so its not only a server feature, so you will allways download all mails new if using a new client without that list, or you changed something on the server ( i.e X-UIDL changed ) and existing list isnt longer build like before see i.e http://kb.mozillazine.org/Popstate.dat however i had some problems with pop3 leave mails on the server with virtual plugin, but that was fixed and works with setup right way -- Best Regards MfG Robert Schetterer From robert at schetterer.org Sat Sep 8 16:12:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:12:53 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: <504B4455.8090600@schetterer.org> Am 08.09.2012 09:39, schrieb Stephan Bosch: > On 9/8/2012 7:11 AM, Dieter Knopf wrote: >> Hello, >> >> is there any update about the sieve command line tool? >> >> I really need some way to refilter my mails without loosings flags and >> without >> moving all messages in a different folder. > > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make > sure your backups are in order before you give it a try and please > report back your findings. > > Regards, > > Stephan. > only for info last time i tried external sieve stuff, it worked with procmail, but i stopped doing something with it -- Best Regards MfG Robert Schetterer From h.reindl at thelounge.net Sat Sep 8 16:16:06 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:16:06 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <504B4516.8040908@thelounge.net> Am 08.09.2012 15:10, schrieb Robert Schetterer: > pop3 feature leave mails on the servers works i.e with thunderbird > with a local list in Tb what is allready download for compare > so its not only a server feature, so you will allways download all mails > new if using a new client without that list, or you changed something > on the server ( i.e X-UIDL changed ) and existing list isnt longer build > like before > > see i.e > > http://kb.mozillazine.org/Popstate.dat > > however i had some problems with pop3 leave mails on the server > with virtual plugin, but that was fixed and works with setup right way and that is why every mail-client has a option "delete messages after x days from server" but users are refusing to spent some minutes of their time looking what options software offers for wondering years later -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From bmalik at ihlas.net.tr Sat Sep 8 16:43:18 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 06:43:18 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <1347111798522-37484.post@n4.nabble.com> i only upgraded dovecot from 1.x to 2.1.x. i didn't change any settings over server. after dovecot service restart, my keep mails on the server, i got again to the outlook. that's my test server but i am going to migrate soon. but i don't want to get duplicate mails. How can i prevent ? Thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html Sent from the Dovecot mailing list archive at Nabble.com. From h.reindl at thelounge.net Sat Sep 8 16:47:41 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:47:41 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347111798522-37484.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B4C7D.108@thelounge.net> Am 08.09.2012 15:43, schrieb bmalik: > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? again: you can NOT POP3 and leave on server depends on the client having a list of message-UUIDs which are already received and it is not uncommon that they are changing due major upgrades on the server DO NOT LEAVE MAILS FOREVER ON THE SERVER this is the only solution and will ever be that is why the client offers "delete after X days from server" smarter clients like thunderbird are defaulting to 14 days not so smart client sleaves their messages forever on the server this is wasting ressources, leads to your troubles and that is why users should learn to look at the options of their sofwtare and try to understand basics who things are working or live with the resulsts if not doing so -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ml at cibest.com Sat Sep 8 18:07:26 2012 From: ml at cibest.com (Cibest ML) Date: Sat, 08 Sep 2012 17:07:26 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120907233844.GA12996@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> Message-ID: <504B5F2E.7070202@cibest.com> On 08/09/2012 01:38, Daniel Parthey wrote: > Cibest ML wrote: >> We've a strange quota behavior on our dovecot servers. >> It seems that it's somehow doubling the real used storage (or could >> be something like twice the inbox size + size of other folders, hard >> to know for sure). >> It was initial setup with the maildir++ system. Now it's been >> configured with dict and mysql table but the result is the same. >> >> Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used >> As example, let's use mailbox at domain.com. >> If I check in the quota table, I'll find : >> - size : 697151354 >> - number of files : 1510 >> >> Now if I check the Maildir folder on the filesystem, I'll find : >> - size : 347960000 >> - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, >> not 100% accurate) >> >> Another mail box >> Reported by dovecot : 503446986 1081 >> Filesystem : 248592000 598 >> >> Does anyone has an idea of what could be possibly wrong. We've >> checked & double-checked everything and nothing seems to be wrong >> (no dsync used). > What happens after quota resync? Does the quota change? > > What happens to the "doveadm quota get" output after LMTP/LDA delivery > of a mail with a fixed-size, let's say one mail with 1 MB? > > How does your setup look like? Mailbox+Director Proxy or only Mailboxes? > > What is your doveconf -n output of mailbox (and possibly director)? > > Regards > Daniel Hi Daniel, Thanks for your answer Here are some tests to show the problem. The test mailbox is initialy empty. All described actions are done in a sequencial way (test mail with an attachement of ~2Mo) - Message delivered by LMTP Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - quota recalc Quota name Type Value Limit User quota STORAGE 5307 524288 User quota MESSAGE 2 - - move to draft + compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move to subfolder "test" in inbox User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move message back to inbox Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - - compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc quota Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - No dovecot director is used at the moment (it's planned). There's one server handling incoming mails (postfix + LMTP), one for IMAP/POP (dovecot) & one for email sending (postfix). dovecot -n output of incoming mails server # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.4.1 Debian 6.0.5 ext4 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_gid = 1500 mail_home = /data/hosting/mail%h/%d/%n mail_location = maildir:/data/hosting/mail%h/%d/%n/Maildir mail_nfs_storage = yes mail_plugins = " quota autocreate" mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Inbox autocreate2 = Trash autocreate3 = Sent autocreate4 = Spam autocreate5 = Drafts autosubscribe = Inbox autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Spam autosubscribe5 = Drafts quota = dict:User quota::noenforcing:proxy::quota quota_rule = *:storage=512M quota_rule2 = Trash:storage=+100M quota_warning = storage=90%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /data/hosting/mail%h/%d/%n/sieve/.dovecot.sieve sieve_before = /etc/dovecot/globalsieve sieve_dir = /data/hosting/mail%h/%d/%n/sieve/ } protocols = " lmtp sieve" quota_full_tempfail = yes service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 10.3.65.71 127.0.0.1 ::1 port = 2055 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } ssl_cert = References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B80A0.9010105@schetterer.org> Am 08.09.2012 15:43, schrieb bmalik: > > > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? > Thanks > dont use pop3, use imap > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- Best Regards MfG Robert Schetterer From haldir.junk at gmail.com Sat Sep 8 23:55:20 2012 From: haldir.junk at gmail.com (Eric Sisolak) Date: Sat, 8 Sep 2012 16:55:20 -0400 Subject: [Dovecot] Vpopmail Dynamic Authentication Module In-Reply-To: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> References: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> Message-ID: On Fri, Aug 31, 2012 at 9:49 AM, Timo Sirainen wrote: > On 24.8.2012, at 16.42, Eric Sisolak wrote: > > > I had to modify the command to build the module, but was able to > > successfully build it: > > gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL > > -I../.. -I../lib -I/home/vpopmail/include -I/home/vpopmail/lib > > userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so > > -L/home/vpopmail/lib/libvpopmail.a > > -L means a path, pointing it to .a file doesn't do anything useful. By > giving the .a file directly probably also fails if libvpopmail.a hasn't > been built with -fPIC. > > > /usr/lib64/dovecot/auth/vpopmail.so: undefined symbol: vclose > > And that's the reason this fails, because libvpopmail isn't included in > vpopmail.so. > > Ah that was it, I was misunderstanding -l and -L. I got it to work with: gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL -I../.. -I../lib -I/home/vpopmail/include -L/home/vpopmail/lib userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so -lvpopmail Thanks! From dieterknopf at googlemail.com Sun Sep 9 06:42:15 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sun, 9 Sep 2012 05:42:15 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: 2012/9/8 Stephan Bosch : > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make sure > your backups are in order before you give it a try and please report back > your findings. Thanks for the info. I'm not really understanding that tool :-( I tried this: sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep But i only get an error: sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' What's the problem? ._REFILTER_ is a standard IMAP subfolder. Thanks From manu at netbsd.org Sun Sep 9 10:17:46 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Sun, 9 Sep 2012 09:17:46 +0200 Subject: [Dovecot] Auth worker max line size In-Reply-To: <19A4E62F-42A0-4FDD-AD68-31C1B26B9E65@iki.fi> Message-ID: <1kq5dlx.15gv4r5j58ya7M%manu@netbsd.org> Timo Sirainen wrote: > Couldn't you change the client to use AUTHENTICATE PLAIN command instead? > The buffer wouldn't be a problem then.. Sorry for the delay, I missed the reply. That is not an option, as the client is not SASL capable. --- src/lib-master/master-auth.h.orig +++ src/lib-master/master-auth.h @@ -13,9 +13,9 @@ /* Authentication client process's cookie size */ #define MASTER_AUTH_COOKIE_SIZE (128/8) /* LOGIN_MAX_INBUF_SIZE should be based on this.*/ -#define MASTER_AUTH_MAX_DATA_SIZE 1024 +#define MASTER_AUTH_MAX_DATA_SIZE 4096 #define MASTER_AUTH_ERRMSG_INTERNAL_FAILURE \ "Internal error occurred. Refer to server log for more information." -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu at netbsd.org From stephan at rename-it.nl Sun Sep 9 11:48:47 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 09 Sep 2012 10:48:47 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: <504AF61A.9090906@rename-it.nl> Message-ID: <504C57EF.4090206@rename-it.nl> On 9/9/2012 5:42 AM, Dieter Knopf wrote: > 2012/9/8 Stephan Bosch : > >> For Pigeonhole v0.3 it is part of the normal release as the sieve-filter >> command. I still haven't received much feedback on it though, so make sure >> your backups are in order before you give it a try and please report back >> your findings. > Thanks for the info. > > I'm not really understanding that tool :-( > > I tried this: > sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep > > But i only get an error: > sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' > > What's the problem? ._REFILTER_ is a standard IMAP subfolder. Mailboxes are not specified in terms of filesystem paths; these are specified in much the same way as for IMAP (albeit in UTF8 where relevant) and doveadm mailbox. So you should try: sieve-filter -u foo at foo.tld main.sieve _REFILTER_ keep The documentation is part of the problem I guess. I've adjusted the man page to specify the mailbox naming explicitly: Regards, Stephan. From roundcube222 at alaadin.org Sun Sep 9 20:51:30 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Sun, 09 Sep 2012 20:51:30 +0300 Subject: [Dovecot] ChgrpNoPerm issue Message-ID: <1b9365b674565d8a794678f3a2858712@Coptics.org> I have dovecot version 2.0.9 I am confused about this article http://wiki2.dovecot.org/Errors/ChgrpNoPerm my mail_location = mbox:~/mymail:INBOX=/var/mail/%u So my inbox is at var/mail/ and my imap folder is /home/user/mymail The problem is /var/mail (Owner is useraccount and the group us mail) , and here comes the problem Dovecot keeps trying to chown the imap folder inside the homedirectory with user:mail account and since And this is the reason of error that appears in my log file. The option mail_access_groups=mail solve the problem.. but I read it is not secure.. With my current configutation, users login with imap to read mail , can they use any the mail_access_groups=mail and read other poeple mail ? does my configutation is a shard mailbox and could be unsecure.. my main quesion , for my current configuration , can i use mail_access_groups=mail safely or can some user use this to read other people mail ? attached my configuration. why didnot have this issue with uw-imapd? and why dovecot try to chown the .imap folder with the mail group ? dovecot -n output is # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <20120909213052.GA11444@daniel.localdomain> Cibest ML wrote: > On 08/09/2012 01:38, Daniel Parthey wrote: > >Cibest ML wrote: > >>We've a strange quota behavior on our dovecot servers. > >>It seems that it's somehow doubling the real used storage (or could > >>be something like twice the inbox size + size of other folders, hard > >>to know for sure). > >>It was initial setup with the maildir++ system. Now it's been > >>configured with dict and mysql table but the result is the same. > Here are some tests to show the problem. The test mailbox is > initialy empty. All described actions are done in a sequencial way > (test mail with an attachement of ~2MB) > > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems ok. > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Strange. There must be some issue with quota recalculation. Maybe there are two instances of the message in the maildir? How large is the actual maildir measured by diskusage (du) command and where are the messages stored in the filesystem? > - move to draft + compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems absolutely correct. Message is copied, deleted and removed from the old directory. > - recalc > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems after compacting the folders there is only one instance of the message. > - move to subfolder "test" in inbox > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > - move message back to inbox > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - The message gets copied and deleted, so there are 2 messages. > - compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - After compaction the "copied+deleted" message gets finally removed. > - recalc quota > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - Very strange. The removed message "reappeared"? Are you using some network filesystem and possibly experience filesystem caching effects? In this case a dovecot director would help. Regards Daniel -- https://plus.google.com/103021802792276734820 From lgb at lgb.hu Mon Sep 10 14:33:08 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Mon, 10 Sep 2012 13:33:08 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" Message-ID: <20120910113307.GG14437@vega.lgb.hu> Hi, I'm trying to use dsync (dovecot version 2.1.9) for IMAP only per-user migration from other IMAP servers to my shiny new one. One worked quite well, but there is a problem with an old IMAP server: dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Looking up IP address dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Connecting to 192.168.10.100:143 dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Disconnected dsync(migtest at office): Error: imapc: Command failed: Disconnected from server dsync(migtest at office): Error: user migtest at office: Initialization failed: Initializing mail storage from mail_location setting failed: imapc: LIST failed: Internal error occurred. Refer to server log for more information. [2012-09-10 11:49:02] dsync(migtest at office): Fatal: User init failed What can cause this? "By hand" test with telnet'ing says: dovecot-test:~$ telnet imap-sunw-old.servers.intra 143 Trying 192.168.10.100... Connected to imap-sunw-old.servers.intra. Escape character is '^]'. * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging Server 6.3-5.02 (built Oct 12 2007; 32bit)) (this is one line of course) Is it possible that the problem about ')' is caused by this line? Or is it tottaly different issue? I just think this, because the the major difference between this and the working server on migration is to have ')' and '(' at the imap "greeting", the working server does not have those, and the error message is about something with ')'. The command was the following (no configuration in dovecot for this, only this command is given at shell level - currently mainly for testing if it works or not); doveadm -D -v -o imapc_features=rfc822.size \ -o imapc_host=192.168.10.100 \ -o imapc_user=migtest at office -o imapc_password=SECRET backup -R -f \ -u migtest at office imapc: This was OK with a dovecot v1 as the "source" (as I've written) but has the problem I've desribed with that "old" sun messaging server as the source. I've also used tcpdump then wireshark's "follow TCP stream" on it: the communication seems to be just that "greeting" line, then my (target) dovecot sends FIN. That "old" server works otherwise since years with various IMAP clients. The log of my server does not contain too much just the db lookups, which works nicely (the same for working and non-working source IMAP servers): Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: prefetch(migtest at office): passdb didn't return userdb entries, trying the next userdb Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): user search: base=cn=mail,dc=office,dc=intra scope=subtree filter=(&(objectClass=mailUser)(uid=migtest at office)) fields=uid Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): result: uid=migtest at office Any help is welcome, - G?bor From ml at cibest.com Mon Sep 10 18:03:39 2012 From: ml at cibest.com (Cibest ML) Date: Mon, 10 Sep 2012 17:03:39 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120909213052.GA11444@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <20120909213052.GA11444@daniel.localdomain> Message-ID: <504E014B.8040309@cibest.com> On 09/09/2012 23:30, Daniel Parthey wrote: > Cibest ML wrote: >> On 08/09/2012 01:38, Daniel Parthey wrote: >>> Cibest ML wrote: >>>> We've a strange quota behavior on our dovecot servers. >>>> It seems that it's somehow doubling the real used storage (or could >>>> be something like twice the inbox size + size of other folders, hard >>>> to know for sure). >>>> It was initial setup with the maildir++ system. Now it's been >>>> configured with dict and mysql table but the result is the same. >> Here are some tests to show the problem. The test mailbox is >> initialy empty. All described actions are done in a sequencial way >> (test mail with an attachement of ~2MB) >> >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems ok. > >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Strange. There must be some issue with quota recalculation. > Maybe there are two instances of the message in the maildir? > How large is the actual maildir measured by diskusage (du) > command and where are the messages stored in the filesystem? > >> - move to draft + compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems absolutely correct. Message is copied, > deleted and removed from the old directory. > >> - recalc >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems after compacting the folders there is > only one instance of the message. > >> - move to subfolder "test" in inbox >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> - move message back to inbox >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > The message gets copied and deleted, > so there are 2 messages. > >> - compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > After compaction the "copied+deleted" > message gets finally removed. > >> - recalc quota >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > Very strange. The removed message "reappeared"? > Are you using some network filesystem > and possibly experience filesystem caching effects? > In this case a dovecot director would help. > > Regards > Daniel The storage is indeed done via NFS. There might be some caching effects but in that case, the behaviour should occur for all the test cases. But it seems only to happen if the mails are stored in INBOX, not in a subfolder of INBOX nor another IMAP folder. We've never been able to reproduce the issue if the email is not exactly in INBOX. We've added some options to check if it's a NFS issue (but it shouldn't be needed as indexes aren't on a NFS filer) : map_disable = yes mail_fsync = always mail_nfs_index = yes It doesn't change anything, same behavior. Here are some test cases to show that there's always 1 mail stored on the filesystem new incoming mail du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota get Quota name Type Value Limit User quota STORAGE 3 524288 User quota MESSAGE 1 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota recacl Quota name Type Value Limit User quota STORAGE 7 524288 User quota MESSAGE 2 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 it really seems to be connected to the recalc operation. Moving to dovecot director would require some work but at the moment, we can't be sure it'd solve the issue (as we only have one IMAP/POP server + one LMTP server, we shouldn't have issues with stickiness of connexions) -- Regards S?bastien From roundcube222 at alaadin.org Mon Sep 10 20:28:37 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 10 Sep 2012 20:28:37 +0300 Subject: [Dovecot] Chown ChgrpNoPerm problem Message-ID: <3d5d646091b201e8545b990fc51c28b8@Coptics.org> Hello, I just have a one simple question. my logs has the following entries Sep 9 02:48:10 dovecot: pop3(r): Error: fchown(/home/r/mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) Sep 9 11:22:30 dovecot: pop3(r): Error: fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) i know that chmod 0600 /var/mail , will solve the problem and i will no longer receive the above errors again But my question is that incase i did not set chmod /var/mail 0600, can i ignore such errors, is these errors harmful ? if this errors keep coming and i ignored them would this cause mbx corruption .. please advise My dovecot conf is disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = hello timo and other dev after a precise reading of plugin deleted to trash it appears to me like things more varariables are declared NULL in the code. it appears to me as being possible to achieve actions even if messages with the same id appears several times in different INBOX when it is the exact order of things -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From ben at indietorrent.org Mon Sep 10 22:19:05 2012 From: ben at indietorrent.org (Ben Johnson) Date: Mon, 10 Sep 2012 15:19:05 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <504E3D29.10102@indietorrent.org> On 9/6/2012 1:56 PM, Ben Johnson wrote: > > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). > >>> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam >> >> That is for version 2.x (note the wiki2)... > > Noted; thank you. > >> I recommend going ahead and upgrading to the latest 2.1.9 and starting >> over. >> > > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle > with this further. > > Thanks again, > > -Ben > So, I setup a development environment that mimics the production environment in question. Must I install Dovecot from source to install Antispam from source? Because I installed Dovecot using a back-ported package ( https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and when I go to install the Antispam plug-in from source, executing the "configure" script yields the following error: # ~/dovecot-antispam-plugin# ./configure [...] checking for dovecot-config in "/usr/local/lib/dovecot"... not found configure: configure: Use --with-dovecot=DIR to provide the path to the dovecot-config file. configure: error: dovecot-config not found I'm assuming that this directory does not exist on my system because I didn't install Dovecot from source. Is this accurate? If so, is my only option to install Dovecot from source and then attempt the same with the Antispam plug-in? Thanks for any pointers, -Ben From silent_wumpus at yahoo.com Tue Sep 11 00:49:12 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Mon, 10 Sep 2012 14:49:12 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <1347313752.51294.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have further read the Migration documentation and someone pointed out to me two good questions: 1 - If I use the modified mb2md.pl -U the program creates a dovecot-uidlist. Or if I can produce this list is there a way I can make dovecot use this list when it creates the dovecot-index file sets the first time it accesses the mailbox? Aka is there a command line option using dovecot or doveadm that creates the indexes based on the dovecot-uidlist that I can script for each mailbox I transfer? 2 - How/where/when do I use "pop3_reuse_xuidl=yes" in the dovecot.conf file for the first time each mailbox is read? Does this option need to be used after the first time (it is not set by default in my new server config). ________________________________ From: Tim E. To: Dovecot List Mailing Sent: Friday, September 7, 2012 3:32 PM Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From jtam.home at gmail.com Tue Sep 11 02:03:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 10 Sep 2012 16:03:46 -0700 (PDT) Subject: [Dovecot] ChgrpNoPerm issue In-Reply-To: References: Message-ID: Robert JR writes: > The problem is /var/mail (Owner is > useraccount and the group us mail) , and here comes the problem > Dovecot > keeps trying to chown the imap folder inside the homedirectory with > user:mail account and since > And this is the reason of error that appears > in my log file. Dovecot is trying to keep the permission of your index caches consistent with that of your mailbox it indexes. The INBOX index cache is kept in your user's mail directory (as per your default settings), although you can change that location. > The option mail_access_groups=mail solve the problem.. > but I read it is not secure.. > With my current configutation, users login > with imap to read mail , can they use > any the mail_access_groups=mail > and read other poeple mail ? Yes, that's the security problem. > does my configutation > is a shard mailbox > and could be unsecure.. Could not parse your question/comment. > why didnot have > this issue with uw-imapd? and why dovecot try to chown the .imap > folder > with the mail group ? uw-imapd was not as picky. The extra consideration for group ownership is so that shared access to mailbox files, and their associated index caches, remain consistent. For example, if you shared a mailbox among your colleagues in group "staff" and the mailbox has group=staff,perm=g+rw, then the index caches will inherit those permissions, and members of group staff can access mailbox and indices alike. [You later write ...] > Sep 9 11:22:30 dovecot: pop3(r): Error: > fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) > failed: Operation not permitted (egid=501(r), group based on > /var/mail/r) > > i know that chmod 0600 /var/mail , will solve the problem > and i will no longer receive the above errors again You also have to make sure that autocreated INBOXs (i.e. a brand new account) does not start out with anything other than 0600. You may have to use dovecot's LDA or twist your LDA's arm to create mailboxes that way. I guess you can also avoid these errors by using memory indices, but you forego the advantages of persistent indices. > But my question is > that incase i did not set chmod /var/mail 0600, can i ignore such > errors, is these errors harmful ? if this errors keep coming and i > ignored them would this cause mbx corruption .. please advise No, you can't ignore these errors. They will break IMAP access to those mailboxes (as you will find out). Joseph Tam From ben at morrow.me.uk Tue Sep 11 04:14:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Tue, 11 Sep 2012 02:14:56 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504E3D29.10102@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> Message-ID: <20120911011456.GA3908@anubis.morrow.me.uk> At 3PM -0400 on 10/09/12 Ben Johnson wrote: > > So, I setup a development environment that mimics the production > environment in question. > > Must I install Dovecot from source to install Antispam from source? > > Because I installed Dovecot using a back-ported package ( > https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and > when I go to install the Antispam plug-in from source, executing the > "configure" script yields the following error: > > # ~/dovecot-antispam-plugin# ./configure > [...] > checking for dovecot-config in "/usr/local/lib/dovecot"... not found > configure: > configure: Use --with-dovecot=DIR to provide the path to the > dovecot-config file. > configure: error: dovecot-config not found > > I'm assuming that this directory does not exist on my system because I > didn't install Dovecot from source. It looks to me like you need to install the dovecot-dev package from that site (if you haven't already), and pass --with-dovecot=/usr/lib/dovecot to configure. Ben From ramon.frontera at uib.es Tue Sep 11 12:41:04 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Tue, 11 Sep 2012 11:41:04 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers Message-ID: Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. We have 2 proxies with director and 4 mail servers. I have configured expire plugin on a single mail server and it works Ok. I don't konw how to configure with 4 mail servers and 2 proxies. My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command Do you have any idea how is the best way to configure expire plugin on this scenario? Thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears From robert at schetterer.org Tue Sep 11 12:58:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 11 Sep 2012 11:58:20 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <504F0B3C.1020105@schetterer.org> Am 11.09.2012 11:41, schrieb Ram?n Frontera: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command > Do you have any idea how is the best way to configure expire plugin on this scenario? > > Thanks in advance. > > -- > Ramon Frontera Gallardo > Universitat de les Illes Balears > > > > > i use one mysql setup for all servers http://wiki2.dovecot.org/Plugins/Expire and doing cron jobs on every server i.e doveadm expunge -A mailbox Trash savedbefore 30d fit the cron times and expunge times and folders to your needs prepare for cron mail messages what got expunged if you have lots off acounts there may be better ways -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Tue Sep 11 15:40:14 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 11 Sep 2012 14:40:14 +0200 Subject: [Dovecot] Is it possible to save only some messages compressed to mdbox? (2.1.6) In-Reply-To: References: <20120820130617.GT24203@ruhr-uni-bochum.de> Message-ID: <20120911124014.GU3002@ruhr-uni-bochum.de> On Thu, Aug 23, 2012 at 11:08:38PM +0300, Timo Sirainen wrote: > > On 20.8.2012, at 16.06, Jost Krieger wrote: > > > So here's the questions: > > > > 1. Is it possible to have compressed/uncompressed mails mixed in an > > mdbox? > > Yes. Compression is detected on a per-mail basis. > > > 2. Will it work via "-o" in lda delivery? > > > > I tried to find out in the source but found it hard to see which options > > are pulled in when. > > -o can override all settings that are handled by the running process itself, so yes that includes compression stuff. Thank you for all your help, but I just can't get it to work. What I have now: For some mailboxes I pt extra parameters in the delivery command (via maildrop): EXTRACOMPRESS="-o debug_log_path=/tmp/lda-debug-log -o mail_debug=yes -o plugin/zlib_save_level=9 -o plugin/zlib_save=bz2" These parameters do arrive in lda: 1. Logging works. 2. If I replace plugin/zlib_save_level=9 with plugin/zlib_save_level=10 I get the correct error message from the zlib plugin. There are no problems in the logs and the mails arrive safely. There is just no compressed material in my mdbox storage files. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From tss at iki.fi Tue Sep 11 16:34:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 16:34:50 +0300 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: On 11.9.2012, at 12.41, Ram?n Frontera wrote: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. From anmeyer at anup.de Tue Sep 11 17:00:38 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:00:38 +0200 Subject: [Dovecot] imap_idle_notify_interval Message-ID: <20120911160038.66c3a387@itx.bitcorner.intern> Hi! I want to change the imap_idle_notify_interval t0 29 minutes and have edited the 20-imap.conf with imap_idle_notify_interval = 29 mins and restarted the server. doveconf -n shows me this: .... protocol imap { imap_idle_notify_interval = 29 mins mail_max_userip_connections = 10 mail_plugins = " quota imap_quota autocreate" } .... doveconf |grep idle shows me the following output: default_idle_kill = 1 mins imap_idle_notify_interval = 2 mins mailbox_idle_check_interval = 30 secs idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 4294967295 secs imap_idle_notify_interval = 29 mins I see two line of imap_idle_notify_interval. What is the right one? Greetings Andreas From tss at iki.fi Tue Sep 11 17:02:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:02:54 +0300 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: <20120911160038.66c3a387@itx.bitcorner.intern> References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: On 11.9.2012, at 17.00, Andreas Meyer wrote: > protocol imap { > imap_idle_notify_interval = 29 mins > mail_max_userip_connections = 10 > mail_plugins = " quota imap_quota autocreate" > } > .... > > doveconf |grep idle shows me the following output: > imap_idle_notify_interval = 2 mins > imap_idle_notify_interval = 29 mins > > I see two line of imap_idle_notify_interval. What is the right one? One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. From anmeyer at anup.de Tue Sep 11 17:06:17 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:06:17 +0200 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: <20120911160617.7bc18f58@itx.bitcorner.intern> Hi! Timo Sirainen wrote: > On 11.9.2012, at 17.00, Andreas Meyer wrote: > > > protocol imap { > > imap_idle_notify_interval = 29 mins > > mail_max_userip_connections = 10 > > mail_plugins = " quota imap_quota autocreate" > > } > > .... > > > > doveconf |grep idle shows me the following output: > > imap_idle_notify_interval = 2 mins > > imap_idle_notify_interval = 29 mins > > > > I see two line of imap_idle_notify_interval. What is the right one? > > One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. So everthing is fine and the imap_idle_notify_interval = 29 mins Thank you! Andreas From tss at iki.fi Tue Sep 11 17:14:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:14:27 +0300 Subject: [Dovecot] ACL MySQL dict: to_user==from_user entries In-Reply-To: <20120829124447.GA16592@sys-241.netcologne.de> References: <20120829124447.GA16592@sys-241.netcologne.de> Message-ID: <217953AF-1EFC-45E3-B5F1-F1BF0938F4FC@iki.fi> On 29.8.2012, at 15.44, Roland Rosenfeld wrote: > But after "SETACL INBOX user at domain lrwstipekxacd" I see a new entry > in my imap_acl_shares table where to_user and from_user both contain > the value "user at domain". > > Does it make sense to run a daily cron job, that removes all entries > with from_user=to_user or may this cause other inconsistencies? Yes. Also fixed in hg: http://hg.dovecot.org/dovecot-2.1/rev/3eaeacc85b4c From tss at iki.fi Tue Sep 11 17:17:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:17:59 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: On 1.9.2012, at 11.10, David Anderson wrote: > I've not found the answer to this question anywhere - please forgive me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. Log errors/warnings to a separate file and watch it. Dovecot's error/warning log typically stays completely empty (except when stopping Dovecot). Anything you find there is a potential error.. > My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Restarting is kind of a kludgy solution, since the only thing it does is to kill all the existing connections and hope that they don't immediately just reconnect back. An equivalent fix is doveadm kick '*' From tss at iki.fi Tue Sep 11 17:24:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:24:46 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50449193.8080101@um.es> References: <50449193.8080101@um.es> Message-ID: On 3.9.2012, at 14.16, Angel L. Mateo wrote: > Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): > > doveadm auth cache flush > 4 cache entries flushed Yep. > So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) Works in my tests. From tss at iki.fi Tue Sep 11 17:31:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:31:16 +0300 Subject: [Dovecot] dovecot proxy ssl-parameter from database? In-Reply-To: <5044B187.6000609@dhag.net> References: <5044B187.6000609@dhag.net> Message-ID: On 3.9.2012, at 16.32, Daniel Funke wrote: > So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) You can't. If disable_plaintext_auth=yes or ssl=required, Dovecot fails the login before any passdb or userdb lookup is done. From tss at iki.fi Tue Sep 11 17:33:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:33:10 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120903141121.GR3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: On 3.9.2012, at 17.11, Jost Krieger wrote: >>> imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) >> >> This looks like it does. Existing emails must not be changed, IMAP >> protocol forbids it (and Dovecot internals don't support it either). >> The only safe way to drop attachments is to assign a new UID for the >> mail, which basically means saving a new message without attachments >> and deleting the old one. The Content-Length header doesn't really >> have anything to do with this error. > > Sorry if I was unclear. This is exactly what the plugin does. It builds > a new message, starting with the old message and leaving out the > unwanted attchment(s). Then it saves the new message, deletes the old > message and finally copies over the flags. > > I had a look at the handling of "Content-Length" in dovecot and had the > impression that a new header was added but the old was not removed (if > formally ok) and on reading the first header wins. But I'm not sure, of > course. Any Content-Length header sent by IMAP client or via LDA input should always get dropped (among some other headers) and replaced with the correct value. So if there's a bug, I doubt it has anything to do with that. But if you can easily reproduce this failure it would be nice to get it fixed.. From ben at indietorrent.org Tue Sep 11 17:40:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 10:40:29 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <20120911011456.GA3908@anubis.morrow.me.uk> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> Message-ID: <504F4D5D.8040207@indietorrent.org> On 9/10/2012 9:14 PM, Ben Morrow wrote: > At 3PM -0400 on 10/09/12 Ben Johnson wrote: >> >> So, I setup a development environment that mimics the production >> environment in question. >> >> Must I install Dovecot from source to install Antispam from source? >> >> Because I installed Dovecot using a back-ported package ( >> https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and >> when I go to install the Antispam plug-in from source, executing the >> "configure" script yields the following error: >> >> # ~/dovecot-antispam-plugin# ./configure >> [...] >> checking for dovecot-config in "/usr/local/lib/dovecot"... not found >> configure: >> configure: Use --with-dovecot=DIR to provide the path to the >> dovecot-config file. >> configure: error: dovecot-config not found >> >> I'm assuming that this directory does not exist on my system because I >> didn't install Dovecot from source. > > It looks to me like you need to install the dovecot-dev package from > that site (if you haven't already), and pass > --with-dovecot=/usr/lib/dovecot to configure. > > Ben > > You nailed it, Ben. Thanks! I downloaded/installed the dovecot-dev package from the same maintainer and was able to build Antispam without issue. The hang-up now seems to be with my configuration. I'm seeing the following error in my email client whenever I attempt to move an email message from, say, Inbox to Junk folder: [SERVERBUG] BUG Unknown internal error. Nothing at all appears in syslog when this happens. I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam , to the letter, I believe. The relevant portions of my configuration are as follows: --------------------- protocol imap { mail_plugins = quota imap_quota antispam } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve antispam_backend = MAILTRAIN antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } --------------------- The permissions and execute bit are set in accordance with other executables in the same directory: # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh I would provide more log content, but the debug options seem to be gone in the forked, >= 2.0 version of this plug-in. Any ideas are very much appreciated. Thanks, -Ben From tss at iki.fi Tue Sep 11 19:02:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:02:33 +0300 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <504647B2.5060804@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> Message-ID: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> On 4.9.2012, at 21.25, Florian Zeitz wrote: > Hello everyone and Timo in particular, > > about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present > in Dovecot. > I had always disliked this a bit, because it replicates a lot of code. > This patch generalizes the HMAC function to take a hash_method struct as > parameter, and changes existing code which uses the "old" HMAC function > to use this new one. > > I'm not really sure this is actually a good idea, but I still felt I > should provide the code in case you would want to merge it upstream. It's otherwise good, but this isn't safe: + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); It assumes that the hmac_init() + hmac_final() is called close to each others. I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) From tss at iki.fi Tue Sep 11 19:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:07:49 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50474093.3010306@um.es> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> Message-ID: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> On 5.9.2012, at 15.07, Angel L. Mateo wrote: > El 05/09/12 13:50, Timo Sirainen escribi?: >> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >> >>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>> >>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>> >>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >> >> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >> > Ok, I'll try this. > > But... is there any way to know how a readdir would be done? What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. From tss at iki.fi Tue Sep 11 19:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:15:37 +0300 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <5048CD0F.9030809@securepoint.de> References: <50471311.2050002@securepoint.de> <5048CD0F.9030809@securepoint.de> Message-ID: <766E466E-3264-42EC-BD35-EEFAE1E2FE17@iki.fi> On 6.9.2012, at 19.19, Dafan Zhai wrote: > I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. > > Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Yes, committed to hg. From tss at iki.fi Tue Sep 11 19:19:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:19:35 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Message-ID: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> On 7.9.2012, at 10.30, Weinreich, Nico wrote: > I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. > > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation This crash was fixed in v2.1.8 I think. > Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. From tss at iki.fi Tue Sep 11 19:20:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:20:02 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: On 7.9.2012, at 13.11, Angel L. Mateo wrote: > Hello, > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > What do you think is the best to activate it? Use dict-file quota instead of Maildir++ quota. From tss at iki.fi Tue Sep 11 19:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:23:22 +0300 Subject: [Dovecot] dovecot/log not running In-Reply-To: <5049DCEC.5050008@hinchfamily.com> References: <5049DCEC.5050008@hinchfamily.com> Message-ID: Something's very wrong. I have no idea what. doveconf -n output might help. On 7.9.2012, at 14.39, Mark Hinch wrote: > I'm running 2.1.7 as downloaded/installed by Debian (and > then appropriately setup for my config, ...). It almost > works. I can send/receive email from thunderbird, squirrelmail, ... > However there are two anomalies: > 1. dovecot/log is not running. After restarting dovecot (and waiting > 30 seconds), I have this in the log files: > > Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 > > After 30 seconds, a "ps -ef | grep dovecot" yields: > > dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login > root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w > mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap > root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot > root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login > dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil > root 25148 20223 0 18:55 ? 00:00:00 dovecot/config > dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth > root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params > mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap > > Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). > A "doveadm log find" dumps this in the log: > Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) > > However, further debugging that I try to do (see next problem) does not result in ANY > further input to the log files, even though I have all known logging turned on. So I *think* > that logging is not actually running. If I can get that started, then I'll take a crack at the > next problem myself ... > > 2. However, as long as I have your attention, my next problem is that it > seems like dovecot/imap-login hangs(?). After a few hours, I have something like: > > # ps -ef | grep dovecot > dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth > root 13277 31186 0 17:47 ? 00:00:00 dovecot/config > dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login > root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot > dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login > root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil > > with exactly 100 imap-login processes. I'm the only meaningful user > on the machine (unless somebody is imap DoS-ing me), so this does not > make much sense. Obviously thunderbird is polling the machine and each > time is leaving around a imap-log process. As far as I can tell, thunderbird > successfully logs in (i.e., I get mail) though I can't be sure whether these > are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? > After a while of course, dovecot will not start any new imap-login processes > so Thunderbird cannot get any more mail until I restart > dovecot. Then everything is OK for another 12 hours or so. > > Ideas on either problem? > > -Mark > From Ralf.Hildebrandt at charite.de Tue Sep 11 19:23:30 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:23:30 +0200 Subject: [Dovecot] FTS indexing Message-ID: <20120911162330.GA25501@charite.de> I'm reacreating all FTS indices on all my users' INBOX folders: mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes doveadm index -u %s -q INBOX In "ps auxwww" I'm seeing this: tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] how can a dovecot/indexer-worker actually access the INBOX of the user "tschuetz" (where "tbudkevy" doesn't have access, due to the different UID)...? Should ~%s/Maildir/lucene-indexes be filled after having run the command sequence for all users? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 19:25:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:25:24 +0300 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <6C88B82F-A2FB-4644-AD0A-CB05A4E6C0ED@iki.fi> On 7.9.2012, at 13.43, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. > > The migration is simple and easy, but there are some configuration settings we cannot set to work properly. > > In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. The only difference should be that you need to switch from Maildir++ quota to dict quota backend. http://wiki2.dovecot.org/Quota/Dict From alec at alec.pl Tue Sep 11 19:29:59 2012 From: alec at alec.pl (A.L.E.C) Date: Tue, 11 Sep 2012 18:29:59 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> Message-ID: <504F6707.1030903@alec.pl> On 09/11/2012 06:19 PM, Timo Sirainen wrote: > The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. This is surprising. Can you provide more info about this issue? -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ben at indietorrent.org Tue Sep 11 19:34:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 12:34:29 -0400 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <504F6815.90407@indietorrent.org> On 9/11/2012 12:23 PM, Timo Sirainen wrote: > Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> >> Not sure on the first issue. On the second issue, out of curiosity, what do you have defined in Thunderbird's Account Settings -> [account in question] -> Server Settings -> Advanced -> Maximum number of server connections to cache? Is this value actually set to 100? -Ben From tss at iki.fi Tue Sep 11 19:38:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:38:17 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911162330.GA25501@charite.de> References: <20120911162330.GA25501@charite.de> Message-ID: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > I'm reacreating all FTS indices on all my users' INBOX folders: > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > how can a dovecot/indexer-worker actually access the INBOX of the > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > different UID)...? indexer-worker runs basically as chroot, but it doesn't bother switching back to root immediately. My guess is that this was a very brief race condition in ps between reading process's UID and the command line. Unless you can easily reproduce this? > Should ~%s/Maildir/lucene-indexes be filled after having run the command > sequence for all users? Only if there was something in the INBOX. Also if users already had previous Lucene indexes it doesn't work, you need to run doveadm fts rescan because dovecot.index remembers which message was last indexed to Lucene. From Ralf.Hildebrandt at charite.de Tue Sep 11 19:54:07 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:54:07 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> Message-ID: <20120911165407.GD25501@charite.de> * Timo Sirainen : > On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > > > I'm reacreating all FTS indices on all my users' INBOX folders: > > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes > > You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? There weren't any. That's why I tried creating them! > > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > > > how can a dovecot/indexer-worker actually access the INBOX of the > > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > > different UID)...? > > indexer-worker runs basically as chroot, but it doesn't bother > switching back to root immediately. My guess is that this was a very > brief race condition in ps between reading process's UID and the > command line. Unless you can easily reproduce this? Can't :) > > Should ~%s/Maildir/lucene-indexes be filled after having run the command > > sequence for all users? > > Only if there was something in the INBOX. My users ALWAYS have something in their inbox :) > Also if users already had previous Lucene indexes it doesn't work, you > need to run doveadm fts rescan because dovecot.index remembers which > message was last indexed to Lucene. OK; I'll try that. Where do I finde the lucene index? I didn't find any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to create the directories. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Tue Sep 11 20:04:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:04:32 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911165407.GD25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> Message-ID: <20120911170432.GE25501@charite.de> * Ralf Hildebrandt : > OK; I'll try that. Where do I finde the lucene index? I didn't find > any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to > create the directories. I fucked up royally: I somehow managed to have fts but not fts_lucene in my list of plugins... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 20:47:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:47:23 +0300 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <20120910113307.GG14437@vega.lgb.hu> References: <20120910113307.GG14437@vega.lgb.hu> Message-ID: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> On 10.9.2012, at 14.33, G?bor L?n?rt wrote: > dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' > * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS > CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO > X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] > imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging > Server 6.3-5.02 (built Oct 12 2007; 32bit)) > > Is it possible that the problem about ')' is caused by this line? I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 From tss at iki.fi Tue Sep 11 20:49:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:49:04 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911170432.GE25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> Message-ID: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> On 11.9.2012, at 20.04, Ralf Hildebrandt wrote: > * Ralf Hildebrandt : > >> OK; I'll try that. Where do I finde the lucene index? I didn't find >> any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to >> create the directories. > > I fucked up royally: I somehow managed to have fts but not fts_lucene > in my list of plugins... It should log errors about that if plugin { fts=lucene } I think? From Ralf.Hildebrandt at charite.de Tue Sep 11 20:51:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:51:45 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> Message-ID: <20120911175145.GJ25501@charite.de> * Timo Sirainen : > > I fucked up royally: I somehow managed to have fts but not fts_lucene > > in my list of plugins... > > It should log errors about that if plugin { fts=lucene } I think? Well, I somehow managed to sprinkle the plugin lines in such a way that no error was triggered. Now I put fts and fts_lucene into mail_plugins and everything works. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From florob at babelmonkeys.de Tue Sep 11 21:07:05 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 11 Sep 2012 20:07:05 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> Message-ID: <504F7DC9.3040408@babelmonkeys.de> Am 11.09.2012 18:02, schrieb Timo Sirainen: > On 4.9.2012, at 21.25, Florian Zeitz wrote: > >> Hello everyone and Timo in particular, >> >> about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present >> in Dovecot. >> I had always disliked this a bit, because it replicates a lot of code. >> This patch generalizes the HMAC function to take a hash_method struct as >> parameter, and changes existing code which uses the "old" HMAC function >> to use this new one. >> >> I'm not really sure this is actually a good idea, but I still felt I >> should provide the code in case you would want to merge it upstream. > > It's otherwise good, but this isn't safe: > > + ctx->ctx = t_malloc(meth->context_size); > + ctx->ctxo = t_malloc(meth->context_size); > > It assumes that the hmac_init() + hmac_final() is called close to each others. I had in fact noticed that. The assumption is currently true for all occurrences, and probably will remain such, but I agree it's better to be safe then sorry. > I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) > Well, either that, or we could use a union of all known context structs there. Possibly plus an i_assert(meth->context_size <= sizeof(union hmac_ctxts)). Or we could use i_malloc() and i_free() under the assumption hmac_init() + hmac_final() calls are always matched. I've a certain preference for the union variant, but it's your call. Regards, Florian From tss at iki.fi Tue Sep 11 21:43:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:43:34 +0300 Subject: [Dovecot] v2.1 memory usage Message-ID: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes? ./configure --with-storages=mdbox sdbox maildir mbox cydir I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :) Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something.. From tss at iki.fi Tue Sep 11 21:48:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:48:08 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <504B5F2E.7070202@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> On 8.9.2012, at 18.07, Cibest ML wrote: > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? From tss at iki.fi Tue Sep 11 21:50:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:50:02 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> On 7.9.2012, at 22.32, Tim E. wrote: > I have a box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Tue Sep 11 23:31:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 23:31:53 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <504F6707.1030903@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> On 11.9.2012, at 19.29, A.L.E.C wrote: > On 09/11/2012 06:19 PM, Timo Sirainen wrote: > >> The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. > > This is surprising. Can you provide more info about this issue? No idea if that's a misconfiguration or what .. or maybe the web browser says it's sending UTF8 but isn't. Anyway, with v2.1.7 it crashes if you send: a SEARCH subject "8bit non-utf8 text" I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation From lgb at lgb.hu Wed Sep 12 00:14:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Tue, 11 Sep 2012 23:14:26 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> References: <20120910113307.GG14437@vega.lgb.hu> <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> Message-ID: <20120911211426.GA15502@vega.lgb.hu> Hi, On Tue, Sep 11, 2012 at 08:47:23PM +0300, Timo Sirainen wrote: [...] > > Is it possible that the problem about ')' is caused by this line? > > I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: I just guessed it was, as the error msg was "server sent invalid input" but the only input from the server according to tcpdump was that line. For sure it's only my oversimplified logic now :) > http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 > http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 Thanks, Timo! With these patches the problem does not show up (honestly, just a quick test after patching/rebuilding, but it seems work now). Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Now I got this, but imho this is totally different issue (mine, not dovcot's) and I will take care about that later. Thanks for your help again! From tss at iki.fi Wed Sep 12 00:29:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 00:29:13 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected > >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected All of these connections had finished handshaking. They simply just seemed to disconnect the connection for no reason. I found one reason for why that could happen, which could explain all of this: http://hg.dovecot.org/dovecot-2.1/rev/24e791bbcf69 The user weakness is remembered until all directors are shut down or all directors have acknowledged the weakness. > >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left The user weakness should have been removed at this point, but looks like there's code missing for it for 1-director-only setup. I'll fix that soon.. > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) These are the weak users causing the trouble. This improves logging: http://hg.dovecot.org/dovecot-2.1/rev/27d3289e1f5c From lgb at lgb.hu Wed Sep 12 01:21:03 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 00:21:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: References: <5049C861.2030904@um.es> Message-ID: <20120911222103.GB15502@vega.lgb.hu> On Tue, Sep 11, 2012 at 07:20:02PM +0300, Timo Sirainen wrote: > On 7.9.2012, at 13.11, Angel L. Mateo wrote: > > > Hello, > > > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > > > What do you think is the best to activate it? > > Use dict-file quota instead of Maildir++ quota. > Sorry to jump into this topic, but can I ask why? Is dict quota is superior over Maildir++ quota in performance in general or it is "only" about the fact, that it's better in the case like above: activating the quota for many users later than the creation of the server itself? The question is interesting for me as well, as I need to implement quota with maildir (on NFS). From tss at iki.fi Wed Sep 12 01:28:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:28:58 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120911222103.GB15502@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> Message-ID: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> On 12.9.2012, at 1.21, G?bor L?n?rt wrote: >>> I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). >>> >>> I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). >>> >>> What do you think is the best to activate it? >> >> Use dict-file quota instead of Maildir++ quota. >> > > Sorry to jump into this topic, but can I ask why? Is dict quota is superior > over Maildir++ quota in performance in general or it is "only" about the > fact, that it's better in the case like above: activating the quota for many > users later than the creation of the server itself? Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. From silent_wumpus at yahoo.com Wed Sep 12 01:39:00 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Tue, 11 Sep 2012 15:39:00 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> Message-ID: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: dsync -v -u user backup user at domain.com All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. I ask this question since the mbox file does not have the associated dovecot indexes. I know Im foobaring/newbying something simple at this point. Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. ________________________________ From: Timo Sirainen To: Tim E. Cc: Dovecot List Mailing Sent: Tuesday, September 11, 2012 2:50 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 7.9.2012, at 22.32, Tim E. wrote: > I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Wed Sep 12 01:48:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:48:53 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Message-ID: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From dmalolepszy at optusnet.com.au Wed Sep 12 08:31:32 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 12 Sep 2012 15:31:32 +1000 Subject: [Dovecot] Strip backslash from usernames Message-ID: <50501E34.2030900@optusnet.com.au> Hi, Is there a way to strip a backslash if it exists at the start of username like \user at example.com within the Dovecot LDA? I am using Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL trickery. Dominic. From alessio at skye.it Wed Sep 12 09:48:19 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 12 Sep 2012 08:48:19 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50503033.9010809@skye.it> Il 12/09/2012 00:28, Timo Sirainen ha scritto: > On 12.9.2012, at 1.21, G?bor L?n?rt wrote: > > >> Sorry to jump into this topic, but can I ask why? Is dict quota is superior >> over Maildir++ quota in performance in general or it is "only" about the >> fact, that it's better in the case like above: activating the quota for many >> users later than the creation of the server itself? > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > But dict quota, for enforce limits, can only be used with dovect as LDA? I'm using Maildir++ with qmail/vpopmail for local delivery and dovecot cannot enforce limits on delivery. With this configuration the only way to update dictionary is "doveadm -A quota rescan". Is correct? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From amateo at um.es Wed Sep 12 10:18:01 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 09:18:01 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> Message-ID: <50503729.1030502@um.es> El 11/09/12 18:07, Timo Sirainen escribi?: > On 5.9.2012, at 15.07, Angel L. Mateo wrote: > >> El 05/09/12 13:50, Timo Sirainen escribi?: >>> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >>> >>>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>>> >>>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>>> >>>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >>> >>> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >>> >> Ok, I'll try this. >> >> But... is there any way to know how a readdir would be done? > > What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. > I meant "when", not how, sorry. But my question is answer with this information. From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:24:28 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:24:28 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:28:51 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:28:51 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0B@s-mcl-mx1.mediacity.drefa.int> Oh sorry, this seems to be an copy&paste failure. "dsr_sub_allocation" was one of the search terms (we run a data warehouse here) ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Timo Sirainen Gesendet: Di 11.09.2012 22:31 An: A.L.E.C Cc: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars [...] I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3837 bytes Desc: not available URL: From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:04:48 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:04:48 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." -searching for a non-UTF-8 term and no mail is found then Roundcube shows correctly a not-found error Until here all is exactly the same, except, that I cannot see the Dovecot crash in /var/log/mail.log anymore. May I assume, that this is now a problem of Roundcube? (But if this is related to Roundcube, why do I still get an connection error (see above)? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Weinreich, Nico Gesendet: Mi 12.09.2012 09:24 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From alec at alec.pl Wed Sep 12 11:16:06 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:16:06 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505044C6.7010504@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: > > -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine > -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:31:08 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:31:08 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl><01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int><01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505044C6.7010504@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0E@s-mcl-mx1.mediacity.drefa.int> Yeah, this fixed it. Will this patch go into source? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von A.L.E.C Gesendet: Mi 12.09.2012 10:16 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. From alec at alec.pl Wed Sep 12 11:37:30 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:37:30 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505049CA.6090202@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: I've found what's really going on here. This is initial search request with utf-8: C: A0003 UID SORT (DATE) UTF-8 ALL OR HEADER SUBJECT {10+} C: L?sung123 HEADER FROM {10+} C: L?sung123 S: * SORT S: A0003 OK Sort completed. Now, because of a bug, fixed in commit I posted before, Roundcube sends another request with search arguments converted to US-ASCII. It should do this only if the first command returns "unsupported charset" response. C: A0004 UID SORT (DATE) US-ASCII ALL OR HEADER SUBJECT {9+} C: L.sung123 HEADER FROM {9+} C: L.sung123 S: A0004 NO Invalid search key The problem is that on that convertion ? is converted to character with code 246 (hex F6), which is from extended ASCII table. This is because in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix this in Roundcube. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ramon.frontera at uib.es Wed Sep 12 11:53:37 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 10:53:37 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure Message-ID: Hello we are running Dovecot 2.1.9. We have 2 proxies with director and 4 mail servers. We have the user information in a LDAP. I try to do: # doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed user mail server ip expire time x.x.x.x 2012-09-12 10:52:17 x.x.x.x 2012-09-12 10:55:53 in the dovecot log i have found: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it What can I do to solve this problem? Thanks in advance. -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears From ml at cibest.com Wed Sep 12 11:58:59 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 12 Sep 2012 10:58:59 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> Message-ID: <50504ED3.5030601@cibest.com> On 11/09/2012 20:48, Timo Sirainen wrote: > On 8.9.2012, at 18.07, Cibest ML wrote: > >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? > Hi, Nothing specific on the namespace side. The only entry is the default one in 10-mail.conf namespace inbox { inbox = yes } I tried to comment it out but it doesn't change anything And no symlinks in the Maildir. Here's the file structure of Maildir for the test mailbox ./dovecot-uidvalidity.504df4e7 ./dovecot-uidvalidity ./.Sent ./.Sent/maildirfolder ./.Sent/new ./.Sent/dovecot.index.log ./.Sent/tmp ./.Sent/dovecot-uidlist ./.Sent/cur ./.Spam ./.Spam/maildirfolder ./.Spam/new ./.Spam/dovecot.index.log ./.Spam/tmp ./.Spam/dovecot-uidlist ./.Spam/cur ./new ./subscriptions ./.Drafts ./.Drafts/maildirfolder ./.Drafts/new ./.Drafts/dovecot.index.log ./.Drafts/tmp ./.Drafts/dovecot-uidlist ./.Drafts/cur ./.Drafts/dovecot-keywords ./dovecot.index.log ./tmp ./dovecot.mailbox.log ./dovecot-uidlist ./.Trash ./.Trash/maildirfolder ./.Trash/new ./.Trash/dovecot.index.log ./.Trash/tmp ./.Trash/dovecot-uidlist ./.Trash/cur ./cur ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa ./dovecot-keywords ./dovecot.index.cache -- Regards S?bastien From alec at alec.pl Wed Sep 12 12:22:27 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 11:22:27 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <505049CA.6090202@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505049CA.6090202@alec.pl> Message-ID: <50505453.4050603@alec.pl> On 09/12/2012 10:37 AM, A.L.E.C wrote: > The problem is that on that convertion ? is converted to character with > code 246 (hex F6), which is from extended ASCII table. This is because > in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix > this in Roundcube. Euh. Of course not PHP issue, the bug was in another place. Fixed. https://github.com/roundcube/roundcubemail/commit/40e1c2d14d11c679d97c00ce831b32d43fad4d32 The fixes are applied to master and release-0.8 branches. ps. sorry for spamming your list, I think it's EOT now. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From amateo at um.es Wed Sep 12 12:54:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 11:54:29 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50505BD5.70907@um.es> El 12/09/12 00:28, Timo Sirainen escribi?: > > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > doveadm quota rescan (not documented)? or doveadm quota recalc (as documented)? Anyway, I'll use dict instead of maildir backend. From mhinch at hinchfamily.com Wed Sep 12 14:13:55 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Wed, 12 Sep 2012 06:13:55 -0500 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <50506E73.30609@hinchfamily.com> Answers for both my helpers - Timo and Ben (Thanks!): Timo: Problem #1 - no dovecot/login doveconf -n output: ============== # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-2-686 i686 Debian wheezy/sid mail_access_groups = mail mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = " imap" ssl_cert = Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> From lgb at lgb.hu Wed Sep 12 14:34:07 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 13:34:07 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <20120912113407.GG22070@vega.lgb.hu> On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: > Activating quota is equally slow, because it needs to scan sizes of all > files. If the size is stored in maildir filename (,S=123) this is pretty > fast, otherwise it's pretty slow because Dovecot stat()s the files, but > afterwards it stores them to dovecot-uidlist file. So the main difference > is that Maildir++ requires rescanning the quota periodically, while dict > quota never rescans unless you run doveadm quota rescan. Ahaa, thanks for your answer. But ... why is rescan needed sometimes with maildir++? If I don't touch maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, not even the LDA, etc), then why is it needed to rescan periodically with maildir? I don't really understand the difference then which needs quota rescan with maildir++ but not with dict quota during the very same usage environment otherwise. Anyway I guess, having dict quota with SQL backend is not so a bad idea: I would be able to check incoming mails (at MXs) if target user will have enough space to hold the mail by using the mail size and quota usage (from SQL what dict quota uses as well) on the MXs using eg postfix policy server ... So I won't generate ugly NDRs later, after accepted mails. Not 100% accurate (quota usage may change while mail hits dovecot) but at least it will stop the majority of those quota related NDRs I have problems with currently. The only thing which keep me away from dict quota with sql backend that it needs much more complex stuff than simply using maildir++ quota, but it's maybe only my lazyness ... thanks again. From ramon.frontera at uib.es Wed Sep 12 16:44:27 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 15:44:27 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <61E579FF-DAB8-4569-BA10-8D599FBD0CA8@uib.es> Hello, I have configured mysql on one server and each mail server use this database for expire plugin. I have configured each mail server as explains on:http://wiki2.dovecot.org/Plugins/Expire But now I have a question; How i configure director server to use the database? I need to configure expire plugin on the director server? thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears El 11/09/2012, a las 15:34, Timo Sirainen escribi?: > On 11.9.2012, at 12.41, Ram?n Frontera wrote: > >> Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. >> We have 2 proxies with director and 4 mail servers. >> >> I have configured expire plugin on a single mail server and it works Ok. >> I don't konw how to configure with 4 mail servers and 2 proxies. >> My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > > The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. > From zinovik.igor at gmail.com Wed Sep 12 16:52:11 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Wed, 12 Sep 2012 17:52:11 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes Message-ID: Hello, list. I'm asking question not directly connected with dovecot. I'm planning a migration from cyrus-imapd to dovecot and i'm using cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes of users, but i'm wondering why size of mailbox in Maildir++ is so much bigger than mailbox in cyrus format after conversion: linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik /srv/vmail/petrsu.ru/z/zinovik/Maildir 238M /mnt/imap/z/user/zinovik 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir I was planning to implement quota for mailboxes about 1 gigabyte, but after conversion I would not be able to receive messages to my own box, because i'm overquota. I think the only way would be to set quota up to 15 GB since some of users have mailbox with 12GB after conversion, but I have no such disk capacity to allow more than 1500 users have 15GB quota. Would it be practical solution to compress all files after conversion and enable zlib support in dovecot? From amateo at um.es Wed Sep 12 17:18:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 16:18:47 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120912113407.GG22070@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> <20120912113407.GG22070@vega.lgb.hu> Message-ID: <505099C7.40709@um.es> El 12/09/12 13:34, G?bor L?n?rt escribi?: > On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: >> Activating quota is equally slow, because it needs to scan sizes of all >> files. If the size is stored in maildir filename (,S=123) this is pretty >> fast, otherwise it's pretty slow because Dovecot stat()s the files, but >> afterwards it stores them to dovecot-uidlist file. So the main difference >> is that Maildir++ requires rescanning the quota periodically, while dict >> quota never rescans unless you run doveadm quota rescan. > > Ahaa, thanks for your answer. > > But ... why is rescan needed sometimes with maildir++? If I don't touch > maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) > just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, > not even the LDA, etc), then why is it needed to rescan periodically with > maildir? I don't really understand the difference then which needs quota > rescan with maildir++ but not with dict quota during the very same usage > environment otherwise. > > Anyway I guess, having dict quota with SQL backend is not so a bad idea: I > would be able to check incoming mails (at MXs) if target user will have > enough space to hold the mail by using the mail size and quota usage (from > SQL what dict quota uses as well) on the MXs using eg postfix policy server > ... So I won't generate ugly NDRs later, after accepted mails. Not 100% > accurate (quota usage may change while mail hits dovecot) but at least it > will stop the majority of those quota related NDRs I have problems with > currently. > > The only thing which keep me away from dict quota with sql backend that it > needs much more complex stuff than simply using maildir++ quota, but it's > maybe only my lazyness ... > You could also use dict quota stored in a file in the user's home directory as maildir++ quota does. This is the way I'm going to use it, not with a sql backend. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From lgb at lgb.hu Wed Sep 12 17:40:00 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:40:00 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb Message-ID: <20120912144000.GB1700@vega.lgb.hu> Hi, I'm about getting to know dovecot in details, I am about the command "doveadm user". The man page shows this example: userdb: jane uid : 8001 gid : 8001 home : /home/jane mail : sdbox:~/sdbox plugins : sieve quota_rule: *:storage=150M It seems to be nice, however I never got this only: ~# doveadm user lgb at office userdb: lgb at office That's all. The dovecot test configuration works otherwise via IMAP/POP3/LMTP nicely. What I guess that the lack of extra information (which would be needed by some scripts of mine, eg for getting user's home easily from shell scripts, as it's hash based, etc) is caused because I don't pass home (and/or other settings) back from userdb and/or passdb. However I do have these in global scope: mail_home = /mailstorage/%Ld/%Ln (yes, this is not hashed at all at the momement but it will be soon) mail_location = maildir:~/Maildir mail_uid = vmail mail_gid = vmail (... and also some global quota stuffs configured in plugin section) Now, I am a bit confused: even with passdb/userdb does not return with user home or GID/UID, dovecot knows these in case of pop3/imap/lmtp access, since these information can be "composed" from those settings. But then, wouldn't it possible so "doveadm user" shows those as well, like lmtp/pop3/imap can resolve those too? Honestly, it's a bit redundant to put needed information into user and pass attrs in every dbs (I have some) when dovecot knows those otherwise ... If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb query tool nothing more), is there any tool which works in this config, displaying extra information (at least user's home), or should I create some script which just repeats the functionality of dovecot's configured mail_home resolution? Thanks. From lgb at lgb.hu Wed Sep 12 17:48:52 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:48:52 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb In-Reply-To: <20120912144000.GB1700@vega.lgb.hu> References: <20120912144000.GB1700@vega.lgb.hu> Message-ID: <20120912144852.GA2372@vega.lgb.hu> On Wed, Sep 12, 2012 at 04:40:00PM +0200, G?bor L?n?rt wrote: > If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb > query tool nothing more), is there any tool which works in this config, > displaying extra information (at least user's home), or should I create some > script which just repeats the functionality of dovecot's configured > mail_home resolution? Arghhh, never mind, it seems switch -m does this :) However this is not documented in the man page, I just discovered because "doveadm user" (without user name) lists that, and I tried out what it does ... From kgc at corp.sonic.net Wed Sep 12 19:55:41 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 12 Sep 2012 09:55:41 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> Message-ID: <5050BE8D.2090002@corp.sonic.net> Timo, I'm glad to hear that you found a plausible cause and have got a patch in to fix it. -K From silent_wumpus at yahoo.com Wed Sep 12 20:51:35 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 10:51:35 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> Message-ID: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From silent_wumpus at yahoo.com Wed Sep 12 22:14:24 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 12:14:24 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> Message-ID: <1347477264.9603.YahooMailNeo@web121506.mail.ne1.yahoo.com> Added these lines to /etc/dovecot.conf namespace { separator = / inbox = yes } Ran the command and it works. However it doesnt keep the UID as the thunderbird client redownloads all the POP3 mails. ________________________________ From: Tim E. To: Dovecot Mailing List Sent: Wednesday, September 12, 2012 1:51 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:38:45 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:38:45 +0200 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <50501E34.2030900@optusnet.com.au> References: <50501E34.2030900@optusnet.com.au> Message-ID: <20120913003845.GA10550@daniel.localdomain> Dominic Malolepszy wrote: > Is there a way to strip a backslash if it exists at the start of > username like \user at example.com within the Dovecot LDA? I am using > Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL > trickery. Address rewriting should be done in the MTA. I see no point why this should be done in the LDA? Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:47:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:47:04 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names Message-ID: <20120913004704.GA10738@daniel.localdomain> Hi, the Drafts special folder is called "Entw?rfe" in German locale. How should the special_use entry in the config file look like? Would you recommended to use the encoded folder name... mailbox "Entw&APw-rfe" { special_use = \Drafts } ...or rather the utf-8 Encoding in the dovecot config file? mailbox "Entw?rfe" { special_use = \Drafts } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:54:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:54:57 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure In-Reply-To: References: Message-ID: <20120913005457.GA10928@daniel.localdomain> Ram?n Frontera wrote: > we are running Dovecot 2.1.9. > We have 2 proxies with director and 4 mail servers. > We have the user information in a LDAP. > > I try to do: > # doveadm director map > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > user mail server ip expire time > x.x.x.x 2012-09-12 10:52:17 > x.x.x.x 2012-09-12 10:55:53 > > in the dovecot log i have found: > dovecot: auth: Error: Trying to iterate users, but userdbs don't support it Please see http://wiki2.dovecot.org/Tools/Doveadm/Mailbox Make sure that the iterate_attrs and iterate_filter settings in /etc/dovecot/dovecot-ldap.conf.ext match your LDAP schema and the user is allowed to iterate the LDAP tree. Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Thu Sep 13 09:34:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 13 Sep 2012 08:34:36 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <50517E7C.5070200@schetterer.org> Am 13.09.2012 02:47, schrieb Daniel Parthey: > Hi, > > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } > > Regards > Daniel > which german client does mailbox "Entw&APw-rfe" and is able to use special_use i ve testet k9 mail 4.2 german, no need for translate \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i think this or equal should be the right way for all clients i ve done this for Archive ( to goal Tb Layout, which hasnt special_use yet), but still untested, so try test something for like this for Drafts too, but for now i dont know a client wich is ready and out for special use with german language ( the new outlook will be, but its not out in german yet , as my latest look on it, the beta is only in englisch or spanish ) mailbox Archive { special_use = \Archive auto=no } mailbox "Archives" { special_use = \Archive auto=subscribe } login telnet . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE SEARCH=FUZZY COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in . LIST (SPECIAL-USE) "" "*" * LIST (\Trash) "/" "Trash" * LIST (\Junk) "/" "Junk" * LIST (\Sent) "/" "Sent" * LIST (\Drafts) "/" "Drafts" * LIST (\Archive) "/" "Archives" * LIST (\Unmarked) "/" "INBOX" -- Best Regards MfG Robert Schetterer From zinovik.igor at gmail.com Thu Sep 13 15:55:00 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Thu, 13 Sep 2012 16:55:00 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes In-Reply-To: References: Message-ID: Completly PEBKAC problem. I was not cleaning Maildir directory after several conversions. 2012/9/12 Igor Zinovik : > Hello, list. > > I'm asking question not directly connected with dovecot. > > I'm planning a migration from cyrus-imapd to dovecot and i'm using > cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes > of users, but i'm wondering why size of mailbox in Maildir++ is so > much bigger than mailbox in cyrus format after conversion: > linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik > /srv/vmail/petrsu.ru/z/zinovik/Maildir > 238M /mnt/imap/z/user/zinovik > 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir > > I was planning to implement quota for mailboxes about 1 gigabyte, > but after conversion I would not be able to receive messages to my own > box, because i'm overquota. > > I think the only way would be to set quota up to 15 GB since some of users > have mailbox with 12GB after conversion, but I have no such disk capacity > to allow more than 1500 users have 15GB quota. > > Would it be practical solution to compress all files after conversion and enable > zlib support in dovecot? From ben at indietorrent.org Thu Sep 13 20:25:18 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 13 Sep 2012 13:25:18 -0400 Subject: [Dovecot] antispam_plugin prevents IMAP login (error 3) [Dovecot 2.0.19] In-Reply-To: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> References: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> Message-ID: <505216FE.20807@indietorrent.org> > On 17.08.2012 12:06, dof at projektfabrik.com wrote: >> Hi everybody, >> >> trying to get the Dovecot antispam_plugin to work and I must be doing >> something wrong, because as soon as it is enabled with a certain >> backend, imap logins do not work anymore (the session is immediately >> closed after a successful login). Interestingly, pipe and spool2dir >> are working (that is, the session won't be closed), dspam-exec and >> crm114-exec are not. If this happens, mail_debug output is as >> follows: >> >> [...] > > Hi again, > > how embarrassing. I should use the correct name for the backend type. > :\ So everything works as expected, sorry for spamming the mailing list. > > > Best regards, > Florian Florian, I am having a very similar (or the same) issue. Ubuntu 12.04.1 LTS with Dovecot 2.0.19. Do you mind sharing the details of your configuration, and what, exactly, you changed with respect to the back-end to resolve the issue? a.) Which back-end are you using? b.) What are your "plugin {}" configuration directives? I'm attempting to use mailtrain, per http://wiki2.dovecot.org/Plugins/Antispam . My configuration directives are: plugin { antispam_backend = mailtrain antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } Thanks for any help! -Ben From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 22:23:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 21:23:08 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <50517E7C.5070200@schetterer.org> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> Message-ID: <20120913192308.GA9400@daniel.localdomain> Robert Schetterer wrote: > > the Drafts special folder is called "Entw?rfe" in German locale. > > How should the special_use entry in the config file look like? > > > > Would you recommended to use the encoded folder name... > > > > mailbox "Entw&APw-rfe" { > > special_use = \Drafts > > } > > > > ...or rather the utf-8 Encoding in the dovecot config file? > > > > mailbox "Entw?rfe" { > > special_use = \Drafts > > } > > which german client does mailbox "Entw&APw-rfe" and is able to use special_use? > > i ve tested k9 mail 4.2 german, no need for translate > \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i > think this or equal should be the right way > for all clients I'm not sure which client created it and if any of my clients does support special_use, but I found the following mailbox on our mdbox filesystem: # ls -ld mail/mailboxes/Entw\&APw-rfe drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe Seems like k9mail or Android Mail must have created it in May 2012, the folder is not subscribed and contains only a single mail. The rest is in "Drafts" mailbox (where it belongs). Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Fri Sep 14 08:33:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 14 Sep 2012 07:33:25 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913192308.GA9400@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> <20120913192308.GA9400@daniel.localdomain> Message-ID: <5052C1A5.40409@schetterer.org> Am 13.09.2012 21:23, schrieb Daniel Parthey: > Robert Schetterer wrote: >>> the Drafts special folder is called "Entw?rfe" in German locale. >>> How should the special_use entry in the config file look like? >>> >>> Would you recommended to use the encoded folder name... >>> >>> mailbox "Entw&APw-rfe" { >>> special_use = \Drafts >>> } >>> >>> ...or rather the utf-8 Encoding in the dovecot config file? >>> >>> mailbox "Entw?rfe" { >>> special_use = \Drafts >>> } >> >> which german client does mailbox "Entw&APw-rfe" and is able to use special_use? >> >> i ve tested k9 mail 4.2 german, no need for translate >> \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i >> think this or equal should be the right way >> for all clients > > I'm not sure which client created it and if any of my clients > does support special_use, but I found the following mailbox on > our mdbox filesystem: > > # ls -ld mail/mailboxes/Entw\&APw-rfe > drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe > > Seems like k9mail or Android Mail must have created it in May 2012, > the folder is not subscribed and contains only a single mail. The > rest is in "Drafts" mailbox (where it belongs). > > Regards > Daniel > I would recommand stay cool until you can verify german clients using imap special_use i dont think it makes sense react now, Timo said he might will create some wiki page in relate clients matrix with special use -- Best Regards MfG Robert Schetterer From gcharot at evenium.com Fri Sep 14 11:59:57 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 10:59:57 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail Message-ID: <5052F20D.7050706@evenium.com> Hello all, I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot of lock and "sieve_execute_bytecode error" in my dovecot.log. $ cat /etc/redhat-release CentOS release 5.8 (Final) $ dovecot --version 1.0.13 The sieve scripts are generated by squirrelmail mail and are mostly mail filters and vacations notices. $ yum info squirrelmail.noarch Installed Packages Name : squirrelmail Arch : noarch Version : 1.4.8 Release : 5.el5.centos.13 When I look at my dovecot.log I have a lot of sieve error : deliver(foo): Sep 11 15:08:04 Info: msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: save failed to EvSUPPORT: Timeout while wait ing for lock deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: Generic Error deliver(foo): Sep 11 15:08:04 Error: sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed The server hosts around 50 mailboxes, some people does have sieve errors some doesn't. Is there any way to get more details about the runtime error ? How can I debug this issue ? Thanks a lot in advance, Greg My config below : #dovecot -n # 1.0.13: /etc/dovecot.conf log_path: /var/log/dovecot.log ssl_ca_file: /etc/pki/roch_crt/roch_evenium_com.ca-bundle ssl_cert_file: /etc/pki/roch_crt/roch_evenium_com.crt ssl_key_file: /etc/pki/roch_crt/private/roch-private.key ssl_cipher_list: ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+SSLv3 login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_process_per_connection: no login_process_size: 256 login_processes_count: 10 login_max_connections: 32 mail_privileged_group: mail dotlock_use_excl: yes fsync_disable: yes maildir_copy_with_hardlinks: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls,oe-ns-eoh auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 From amateo at um.es Fri Sep 14 12:13:03 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 14 Sep 2012 11:13:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049CC5D.7070005@schetterer.org> References: <5049C861.2030904@um.es> <5049CC5D.7070005@schetterer.org> Message-ID: <5052F51F.3040605@um.es> El 07/09/12 12:28, Robert Schetterer escribi?: >> >> I have tried to activate it in one of my nodes. The problem is that >> load of it has increase a lot, as much as the system was unusable >> (maildir is in nfs storage, with indexes in local disks). > > that might not be the optimal storage solution > Why do you say it might be not optimal? I think that nfs storage is a very common scenario. From carsten.delellis at delellis.net Fri Sep 14 12:27:23 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Fri, 14 Sep 2012 11:27:23 +0200 Subject: [Dovecot] Couldn't drop privileges: User is missing UID (see mail_uid setting) Message-ID: Hi all I hope anyone could help me out with my problem regarding the user_id settings. I am running my mail servers just for fun and I am not a professional mail admin. After I am using postfix and dovecot for my mail servers with a mysql backend for the last 2 years I have tried to set up an installation with a ldap backend. I followed the how-to from the ?Postfix Book? and now I have always the following error message: Couldn't drop privileges: User is missing UID (see mail_uid setting) This error message occurs whenever I try to connect via a mail client. System emails were delivered the way I expected. So I assume postfix is working fine. The ldap search I use for user authentication is as follows: uris = ldap://myserver.com dn = cn=admin, dc=mydomain, dc=com dnpass = secure tls = no base = ou=mailuser,dc=mydomain,dc=com ldap_version = 3 user_attrs = pfMailMessageStore=home user_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) pass_attrs = pfClearPassword=password,pfMailMessageStore=userdb_home pass_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) default_pass_scheme = PLAIN user_global_uid = 5000 user_global_gid = 5000 I am using dovecot 2.1.9, postfix 2.8.5 and openldap 2.4.23 I would appreciate any help or tipps. ~ Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From stephan at rename-it.nl Fri Sep 14 12:48:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 11:48:21 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052F20D.7050706@evenium.com> References: <5052F20D.7050706@evenium.com> Message-ID: <5052FD65.8020104@rename-it.nl> Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: > Hello all, > > I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot > of lock and "sieve_execute_bytecode error" in my dovecot.log. > > $ cat /etc/redhat-release > CentOS release 5.8 (Final) > > $ dovecot --version > 1.0.13 Very, very, very, very old. > The sieve scripts are generated by squirrelmail mail and are mostly > mail filters and vacations notices. > > $ yum info squirrelmail.noarch > Installed Packages > Name : squirrelmail > Arch : noarch > Version : 1.4.8 > Release : 5.el5.centos.13 > > When I look at my dovecot.log I have a lot of sieve error : > > deliver(foo): Sep 11 15:08:04 Info: > msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: > save failed to EvSUPPORT: Timeout while wait > ing for lock There's the actual error. This has very little to do with Sieve itself. Looks like some mbox locking issue, but I don't really know. You could try enabling mail_debug to get more info. > deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: > Generic Error > deliver(foo): Sep 11 15:08:04 Error: > sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed Yuck, I was hoping that people would finally have stopped using cmusieve. Regards, Stephan. From Jost.Krieger+dovecot at rub.de Fri Sep 14 13:47:59 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 14 Sep 2012 12:47:59 +0200 Subject: [Dovecot] Possible mbox corruption In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: <20120914104758.GU30629@ruhr-uni-bochum.de> On Tue 11 Sep 2012 05:33:10 PM GMT, Timo Sirainen wrote: > Any Content-Length header sent by IMAP client or via LDA input should > always get dropped (among some other headers) and replaced with the > correct value. So if there's a bug, I doubt it has anything to do with > that. But if you can easily reproduce this failure it would be nice to > get it fixed.. You're right, of course. And I cannot reproduce the problem. The user that could reproduce it has somehow given up. I'v experimented and read the source, and there is no way a "Content-Length" could be saved by dovecot :-) But ... This is on a system where delivery is not (yet) via dovecot, but directly by qmail. Two possibilities come to mind: 1. We have a locking problem that only surfaces when qmail and dovecot try to write at the same time to the end of the mbox. Most people (and clients) hardly ever do an APPEND to the INBOX. 2. Something qmail writes to the INBOX makes dovecot hiccup (like a wrong Content-Length header :-;). Which would anyone think more likely? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From gcharot at evenium.com Fri Sep 14 14:03:25 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 13:03:25 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052FD65.8020104@rename-it.nl> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> Message-ID: <50530EFD.8080907@evenium.com> Thanks for the quick reply Stephan ! Is it possible to upgrade from 1.0.13 (which is the official CentOS release) to the atrpms version 2.1.1-2 available at http://packages.atrpms.net/dist/el5/dovecot/ Do I need a migration plan ? Thanks ! Greg Le 14/09/2012 11:48, Stephan Bosch a ?crit : > Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: >> Hello all, >> >> I'm using dovecot on CentOS 5.8 (official CentOS package) and have >> lot of lock and "sieve_execute_bytecode error" in my dovecot.log. >> >> $ cat /etc/redhat-release >> CentOS release 5.8 (Final) >> >> $ dovecot --version >> 1.0.13 > > Very, very, very, very old. > >> The sieve scripts are generated by squirrelmail mail and are mostly >> mail filters and vacations notices. >> >> $ yum info squirrelmail.noarch >> Installed Packages >> Name : squirrelmail >> Arch : noarch >> Version : 1.4.8 >> Release : 5.el5.centos.13 >> >> When I look at my dovecot.log I have a lot of sieve error : >> >> deliver(foo): Sep 11 15:08:04 Info: >> msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: >> save failed to EvSUPPORT: Timeout while wait >> ing for lock > > There's the actual error. This has very little to do with Sieve > itself. Looks like some mbox locking issue, but I don't really know. > You could try enabling mail_debug to get more info. > >> deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: >> Generic Error >> deliver(foo): Sep 11 15:08:04 Error: >> sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed > > Yuck, I was hoping that people would finally have stopped using cmusieve. > > Regards, > > Stephan. From stephan at rename-it.nl Fri Sep 14 14:37:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 13:37:07 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <50530EFD.8080907@evenium.com> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> <50530EFD.8080907@evenium.com> Message-ID: <505316E3.5010802@rename-it.nl> Op 9/14/2012 1:03 PM, Gregory Charot (EVENIUM) schreef: > Thanks for the quick reply Stephan ! > > Is it possible to upgrade from 1.0.13 (which is the official CentOS > release) to the atrpms version 2.1.1-2 available at > > http://packages.atrpms.net/dist/el5/dovecot/ > > Do I need a migration plan ? http://wiki2.dovecot.org/Upgrading?action=show&redirect=UpgradingDovecot http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Migration Regards, Stephan. From roundcube222 at alaadin.org Fri Sep 14 13:37:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 14 Sep 2012 13:37:31 +0300 Subject: [Dovecot] POP3S PORT Message-ID: Hello I am using dovecot with POP3S and IMAPS ports, When client connect to POP3S port (995) , i can see in dovecot log file that the protocol is TLSv1 So how come the protocol is TLSv1 and not SSL ? Does the user connect t port 995 with SSL protocol then encrypt again inside the SSL session to make The TLSv1 protocol ? Or does the client use TLSv1 Protocol immediately when connect to port 995 without Using SSL protocol ? Please Help. Robert JR From mailinglist at darac.org.uk Fri Sep 14 14:49:15 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 14 Sep 2012 12:49:15 +0100 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <20120914114915.GA23431@darac.org.uk> On Fri, Sep 14, 2012 at 01:37:31PM +0300, Robert JR wrote: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? First of all, read http://wiki2.dovecot.org/SSL and see if that answers your question. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 14 15:06:32 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 14 Sep 2012 14:06:32 +0200 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <50531DC8.207@thelounge.net> Am 14.09.2012 12:37, schrieb Robert JR: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? because it is simply the same read about what TLS is -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From damoxc at gmail.com Fri Sep 14 17:46:39 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 15:46:39 +0100 Subject: [Dovecot] key -> object mailstore Message-ID: Hi, I was wondering what would be entailed in modifying dovecot 2.2 to support storing mail in an object store. I've seen a few mails dotted around in the ML history about supporting such a thing and seen it's basically dependant upon some changes in lib-storage to support writing messages without locking. Is this still the case? Regards, Damien From tss at iki.fi Fri Sep 14 17:59:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 14 Sep 2012 17:59:17 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: Message-ID: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> On 14.9.2012, at 17.46, Damien Churchill wrote: > I was wondering what would be entailed in modifying dovecot 2.2 to > support storing mail in an object store. I've seen a few mails dotted > around in the ML history about supporting such a thing and seen it's > basically dependant upon some changes in lib-storage to support > writing messages without locking. Is this still the case? I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? The generic idea is: - only one server accesses one user simultaneously - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync - support high latency: asynchronous reads/writes. prefetch mail bodies. From damoxc at gmail.com Fri Sep 14 18:16:09 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 16:16:09 +0100 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: On 14 September 2012 15:59, Timo Sirainen wrote: > On 14.9.2012, at 17.46, Damien Churchill wrote: > >> I was wondering what would be entailed in modifying dovecot 2.2 to >> support storing mail in an object store. I've seen a few mails dotted >> around in the ML history about supporting such a thing and seen it's >> basically dependant upon some changes in lib-storage to support >> writing messages without locking. Is this still the case? > > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? > That's good to hear :) I've been evaluating RADOS as an object store, which is similar to S3. Although any distributed storage would be great. I'd be more than happy to help code it! > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. > I'm assuming that the director would be used in order to distribute connections to the same server, so it's only within a local instance of dovecot you'd need to be aware of what currently has a connection open for that user? How are you planning on handling the situation where say node X dies and hasn't uploaded the latest index file? Would that result in missing messages from the mailbox when accessed by another node, or is the local index intended to be more of a write-through cache? From ncjeffgus at zimage.com Sat Sep 15 06:39:36 2012 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Fri, 14 Sep 2012 20:39:36 -0700 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <1347680376.5717.61.camel@maclinux> On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it > for v2.2. Do you want to help coding it? :) Which storage would you > want to use? > > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and > accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split > brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. With this system, would the read/write ultimately go to a normal OS file function? If it is a file function, could this be used with a system like glusterfs, ceph, etc? The other option would be to write it against a object store client library and bypass the normal file functions. ...Jeff From patrickdk at patrickdk.com Sat Sep 15 19:50:47 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sat, 15 Sep 2012 12:50:47 -0400 Subject: [Dovecot] v2.1 memory usage In-Reply-To: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> References: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> Message-ID: <20120915125047.Horde.iNAEcZLnE6FQVLHnJ7HDDyA@mail.patrickdk.com> Only have 650 imap clients this weekend, have to give it a try during a weekday. Total(MB) Avg(kB) Name 2 4 /lib/ld-2.11.1.so 2 4 /lib/libbz2.so.1.0.4 2 4 /lib/libc-2.11.1.so 63 96 /lib/libcrypto.so.0.9.8 2 4 /lib/libdl-2.11.1.so 0 0 /lib/libnss_dns-2.11.1.so 0 0 /lib/libnss_files-2.11.1.so 2 4 /lib/libpthread-2.11.1.so 0 0 /lib/libresolv-2.11.1.so 2 4 /lib/librt-2.11.1.so 13 20 /lib/libssl.so.0.9.8 2 4 /lib/libz.so.1.2.3.3 2 4 /usr/lib/dovecot/imap 10 16 /usr/lib/dovecot/libdovecot-storage.so.0.0.0 2 4 /usr/lib/dovecot/libdovecot.so.0.0.0 2 4 /usr/lib/dovecot/modules/lib10_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib20_zlib_plugin.so 2 4 /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so 435 661 [anonymous] 640 973 [heap] 55 84 [stack] 1253 58352 TOTAL Quoting Timo Sirainen : > v2.1's imap/pop3 processes are linked with OpenSSL by default, > because of imapc/pop3c storages. I wonder if you notice any actual > (system-global) memory usage reduction when compiling without those > and you have thousands of imap processes? > > ./configure --with-storages=mdbox sdbox maildir mbox cydir > > I guess I could try it myself also but maybe someone else wanting to > reduce memory usage is quicker? :) > > Related to that, you can look at where imap processes' memory goes > with: http://dovecot.org/tools/linux-maps-parse-imap.pl > I don't know how much memory those (writable!) file-backed memory > areas actually use per-process. Maybe nothing, maybe something.. From carsten.delellis at delellis.net Sun Sep 16 13:48:17 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 12:48:17 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database Message-ID: Hi all I receive the following error in my mail.log. Does anybody know how to configure dovecot to get rid of this ? I have one server up and running with postfix, dovecot and a mysql backend. The error above happens on another server with postfix, dovecot and a ldap backend. Any hints ? Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From user+dovecot at localhost.localdomain.org Sun Sep 16 14:21:11 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Sun, 16 Sep 2012 13:21:11 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <5055B627.9010809@localhost.localdomain.org> On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > Hi all > > I receive the following error in my mail.log. > > Does anybody know how to configure dovecot to get rid of this ? > [?] Fix your setup. ,--[ http://wiki2.dovecot.org/UserDatabase ]-- | The user database lookup returns these fields: | ? | * home: User's home directory, overrides the global mail_home setting. | ? `-- See also: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Pascal -- The trapper recommends today: cafefeed.1226013 at localdomain.org From noel.butler at ausics.net Sun Sep 16 14:32:01 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:32:01 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <5055B627.9010809@localhost.localdomain.org> References: <5055B627.9010809@localhost.localdomain.org> Message-ID: <1347795121.3768.5.camel@tardis> thats a rather intelligent response now, isnt it troll. On Sun, 2012-09-16 at 13:21 +0200, Pascal Volk wrote: > On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > > Hi all > > > > I receive the following error in my mail.log. > > > > Does anybody know how to configure dovecot to get rid of this ? > > [?] > > Fix your setup. > > ,--[ http://wiki2.dovecot.org/UserDatabase ]-- > | The user database lookup returns these fields: > | ? > | * home: User's home directory, overrides the global mail_home setting. > | ? > `-- > > See also: http://wiki2.dovecot.org/VirtualUsers/Home > > > Regards, > Pascal -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Sun Sep 16 14:33:33 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:33:33 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <1347795213.3768.8.camel@tardis> Hi, On Sun, 2012-09-16 at 12:48 +0200, Carsten Laun-De Lellis wrote: > Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a ldap > backend. should use mysql :) -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From mostafa.rokooie at gmail.com Sun Sep 16 16:44:49 2012 From: mostafa.rokooie at gmail.com (Mostafa Rokooie) Date: Sun, 16 Sep 2012 18:14:49 +0430 Subject: [Dovecot] Edit an existing message or replacing it Message-ID: Hi, I want to know is there any solution to edit an existing message in mailbox via IMAP? or replace it with a new message (store a new message with same UID)? Thank you very much, --Mostafa Rokooie From carsten.delellis at delellis.net Sun Sep 16 17:30:36 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 16:30:36 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <1347795213.3768.8.camel@tardis> References: <1347795213.3768.8.camel@tardis> Message-ID: Hi First of all thank you for helping me to fix the problem. I am trying out ldap to get used to it. As I wrote I have a mysql setup up and running. Here's the output of doveconf -n: # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-028stab099.3 x86_64 Ubuntu 11.04 vzfs auth_mechanisms = plain login disable_plaintext_auth = no hostname = smtp.example.com mail_location = maildir:/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/vmail/%d/%n prefix = separator = / subscriptions = yes type = private } passdb { args = /scripts/ldap/mail/auth.ldap driver = ldap } plugin { recipient_delimiter = + sieve = /vmail/%d/%n/dovecot.sieve sieve_dir = /vmail/%d/%n sieve_global_dir = /var/lib/dovecot/sieve sieve_global_path = /var/lib/dovecot/sieve/default.sieve } postmaster_address = root at example.com protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0660 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl_cert = Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a > ldap backend. should use mysql :) From CMarcus at Media-Brokers.com Sun Sep 16 17:41:10 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 16 Sep 2012 10:41:10 -0400 Subject: [Dovecot] clucene build warnings? Message-ID: <5055E506.7040704@Media-Brokers.com> Hi all/Timo, Is clucene still the best/recommended method for enabling full text search? Although I haven't enabled FTS yet, I plan on doing sometime in the future, and was going to be using clucene, but before I do, I'd like to see if I need to worry about an issue I've been seeing ever since I first installed clucene (and still get every time it is upgraded)... I run gentoo linux on an amd64 server, and have always gotten this warning whenever it is updated (and also got it when I first installed it): QA Notice: Package triggers severe warnings which indicate that it may exhibit random runtime failures. /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? Please do not file a Gentoo bug and instead report the above QA issues directly to the upstream developers of this software. Homepage:http://clucene.sourceforge.net/ Has anyone else ever seen similar warnings when building clucene? Is the gentoo ebuild output correct that this is not a gentoo bug, but is in fact a problem with upstream? If so, is it harmless? Or should I try to get upstream to fix it? That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? Thanks, -- Best regards, Charles From sven at svenhartge.de Sun Sep 16 18:20:48 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 16 Sep 2012 17:20:48 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database References: <1347795213.3768.8.camel@tardis> Message-ID: <6947fe6hfsv8@mids.svenhartge.de> Carsten Laun-De Lellis wrote: > user_attrs = home=/vmail/%d/%n,=uid=5000,=gid=5000 This should be user_attrs = =home=/vmail/%d/%n,=uid=5000,=gid=5000 You need the = before "home" like you did with "uid" and "gid" because you are not assigning any LDAP attribute to home but are construction one yourself with %d and %n. Or you can use the default_fields stanza in your userdb configuration to provide those values. user_attrs would then be empty. Gr??e, Sven. -- Sigmentation fault. Core dumped. From lists at kokelnet.de Sun Sep 16 18:48:28 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Sun, 16 Sep 2012 17:48:28 +0200 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. Message-ID: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Hello list, I am using dovecot v2.1.9 and have a problem with doveadm backup: # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here my namespace config: namespace { list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/shared/%%u prefix = Gemeinsame-Postfaecher/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u prefix = separator = / subscriptions = yes type = private } Have I got a problem with my namespace config? Regards, Tobias Hachmer From slusarz at curecanti.org Sun Sep 16 21:55:39 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 16 Sep 2012 12:55:39 -0600 Subject: [Dovecot] Edit an existing message or replacing it In-Reply-To: References: Message-ID: <20120916125539.Horde.bh8AKIF5lbhQViCrQKfX2pA@bigworm.curecanti.org> Quoting Mostafa Rokooie : > I want to know is there any solution to edit an existing message in mailbox > via IMAP? or replace it with a new message (store a new message with same > UID)? You **can't** alter an existing UID message. You have to re-add the message to the mailbox and use a new UID. For this, you have 2 options: 1.) FETCH the message, alter it locally, APPEND the message to the server, and DELETE the original message. 2.) Use CATENATE to create a new message, and DELETE the original message (this is how we strip attachments from a message). Note that not all IMAP4rev1 servers support CATENATE. michael From dmalolepszy at optusnet.com.au Mon Sep 17 03:49:25 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 17 Sep 2012 10:49:25 +1000 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <20120913003845.GA10550@daniel.localdomain> References: <50501E34.2030900@optusnet.com.au> <20120913003845.GA10550@daniel.localdomain> Message-ID: <50567395.1040001@optusnet.com.au> On 13/09/12 10:38, Daniel Parthey wrote: > Dominic Malolepszy wrote: >> Is there a way to strip a backslash if it exists at the start of >> username like \user at example.com within the Dovecot LDA? I am using >> Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL >> trickery. > > Address rewriting should be done in the MTA. > I see no point why this should be done in the LDA? > > Regards > Daniel Agreed, I was just hoping I would not have to poke around the existing Sendmail setup we have here, in the end it turned out the backslashes were leftover legacy crud in some of the LDAP entries that actually did nothing. From crohmann at netcologne.de Mon Sep 17 09:52:37 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Mon, 17 Sep 2012 08:52:37 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <5056C8B5.3070706@netcologne.de> Hey dovecot-users, On 14.09.2012 16:59, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? I'd vote for OpenStack's Swift or Ceph's RADOS. They are both gaining momentum with new instrallations, they are open source and quite active in development. Also they both maintain Amazon S3 compatible APIs. Ceph even has a Swift compatible API for that matter. Regards Christian From alessio at skye.it Mon Sep 17 10:09:09 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 17 Sep 2012 09:09:09 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment Message-ID: <5056CC95.8030209@skye.it> Hi, I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. Dovecot's wiki suggest to enable acl like this: plugin { # assumes mailboxes are in /var/mail/%d/%n: acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db } but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. So, why don't provide a way to restrict shared mailboxes also for dict in SQL? One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. Are there other possible solutions? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tom at whyscream.net Mon Sep 17 11:40:11 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 17 Sep 2012 10:40:11 +0200 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <5056E1EB.5000600@whyscream.net> On 9/16/12 4:41 PM, Charles Marcus wrote: > Hi all/Timo, > > Is clucene still the best/recommended method for enabling full text search? > > Although I haven't enabled FTS yet, I plan on doing sometime in the > future, and was going to be using clucene, but before I do, I'd like to > see if I need to worry about an issue I've been seeing ever since I > first installed clucene (and still get every time it is upgraded)... > > I run gentoo linux on an amd64 server, and have always gotten this > warning whenever it is updated (and also got it when I first installed it): > > QA Notice: Package triggers severe warnings which indicate that it > may exhibit random runtime failures. > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: > warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, > int, size_t)? > Please do not file a Gentoo bug and instead report the above QA > issues directly to the upstream developers of this software. > Homepage:http://clucene.sourceforge.net/ > > Has anyone else ever seen similar warnings when building clucene? > > Is the gentoo ebuild output correct that this is not a gentoo bug, but > is in fact a problem with upstream? > > If so, is it harmless? Or should I try to get upstream to fix it? Same warnings here (gentoo 32bit). The gentoo package management simply checks the gcc output for warnings, and reports them to you after the build completes. As long as the warnings are not related to patches added by gentoo, the issue should go upstream. Whether it is a serious issue, that can only be decided by reviewing the code itself, which would be more appropriate for a lucene dev than a dovecot dev ;) I suggest that you simply report the issue upstream and see what happens. If the code is harmless, they might just change it the so that gcc doesn't complain about it in the future. -- Tom From CMarcus at Media-Brokers.com Mon Sep 17 12:56:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 05:56:35 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5056E1EB.5000600@whyscream.net> References: <5055E506.7040704@Media-Brokers.com> <5056E1EB.5000600@whyscream.net> Message-ID: <5056F3D3.4060006@Media-Brokers.com> On 2012-09-17 4:40 AM, Tom Hendrikx wrote: > I suggest that you simply report the issue upstream and see what > happens. If the code is harmless, they might just change it the so that > gcc doesn't complain about it in the future. Thanks Tom, but I already did, a long time ago (when I first installed it): https://sourceforge.net/tracker/?func=detail&aid=3494798&group_id=80013&atid=558446 Didn't hear anything for a month, and when I posted a follow-up, was told they were busy, and would take a look in the next few days, but nothing since and that was back in March... I just posted another follow-up, so we'll see what happens. I was mainly interested to know if anyone else sees this warning when compiling on other systems... -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 13:02:51 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:02:51 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) Message-ID: Hello, I have a quite common dovecot configuration with virtual users support: 1. User configuration is stored in MySQL database. 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. Is it possible to change mailbox mode to 0660? Dovecot version is 2.1.7 Part of SQL configuration: user_query = \ SELECT \ CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home, \ 500 AS uid, \ 500 AS gid \ FROM users u \ LEFT JOIN domains d ON u.domain_id = d.id \ WHERE \ u.user_name = '%n' AND \ d.domain_name = '%d' AND \ u.active = 1 Regards. -- M. From marcin at rootnode.net Mon Sep 17 13:09:22 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:09:22 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <2c3a5a4b367a9062389dfeb36d2cba61@rootnode.net> On 2012-09-17 12:02, Marcin H?ybin wrote: > 1. User configuration is stored in MySQL database. Obviously, I meant 0700 to be changed to 0770. -- M. From tss at iki.fi Mon Sep 17 15:21:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:21:51 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> On 16.9.2012, at 17.41, Charles Marcus wrote: > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? That's wrong, but it works anyway as intended. > That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? CLucene seems to be a pretty dead project. They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. From tss at iki.fi Mon Sep 17 15:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:36:32 +0300 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <55461F54-DC83-476B-910C-D8783E675DE8@iki.fi> On 13.9.2012, at 3.47, Daniel Parthey wrote: > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } UTF8 is (or should be) now used everywhere in config files. From tss at iki.fi Mon Sep 17 15:40:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:40:15 +0300 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. In-Reply-To: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> References: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Message-ID: <50602815-DD35-4B6B-B319-AA9261AC0812@iki.fi> On 16.9.2012, at 18.48, Tobias Hachmer wrote: > I am using dovecot v2.1.9 and have a problem with doveadm backup: > > # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir > doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Different installations use different kinds of namespaces for different purposes, so dsync can't simply sync all the namespaces. It syncs only the namespace which uses the global mail_location setting (not overriding it by having an explicit location setting). > namespace inbox { > inbox = yes > location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u So you probably should remove this location setting (and make sure mail_location is the same). > prefix = > separator = / > subscriptions = yes > type = private > } From CMarcus at Media-Brokers.com Mon Sep 17 15:42:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 08:42:32 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> Message-ID: <50571AB8.3010505@Media-Brokers.com> On 2012-09-17 8:21 AM, Timo Sirainen wrote: > CLucene seems to be a pretty dead project. Bummer... Was my memory correct that lucene is what you currently recommend for FTS for dovecot? > They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. Yuck - looks like it is java based? http://lucene.apache.org/ -- Best regards, Charles From tss at iki.fi Mon Sep 17 15:44:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:44:21 +0300 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <5056CC95.8030209@skye.it> References: <5056CC95.8030209@skye.it> Message-ID: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> On 17.9.2012, at 10.09, Alessio Cecchi wrote: > I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. > > Dovecot's wiki suggest to enable acl like this: > > plugin { > # assumes mailboxes are in /var/mail/%d/%n: > acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db > } > > but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. As long as all the servers have access to the file it doesn't matter where it is. You could even do something like /var/mail/nas1/shared-dict/%d.db > So, why don't provide a way to restrict shared mailboxes also for dict in SQL? > > One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. From tss at iki.fi Mon Sep 17 15:45:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:45:28 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50571AB8.3010505@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> Message-ID: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> On 17.9.2012, at 15.42, Charles Marcus wrote: > On 2012-09-17 8:21 AM, Timo Sirainen wrote: >> CLucene seems to be a pretty dead project. > > Bummer... > > Was my memory correct that lucene is what you currently recommend for FTS for dovecot? Lucene or Solr (i.e. anything but Squat). >> They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. > > Yuck - looks like it is java based? > > http://lucene.apache.org/ Both CLucene and Lucene++ are C++ ports of the original Java Lucene. From tss at iki.fi Mon Sep 17 15:46:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:46:38 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> On 17.9.2012, at 13.02, Marcin H?ybin wrote: > I have a quite common dovecot configuration with virtual users support: > 1. User configuration is stored in MySQL database. > 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. > > Is it possible to change mailbox mode to 0660? Yeah, although since they're already sharing UIDs I don't see much point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway describes how it all works. From tss at iki.fi Mon Sep 17 15:57:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:57:57 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> On 14.9.2012, at 18.16, Damien Churchill wrote: > On 14 September 2012 15:59, Timo Sirainen wrote: >> On 14.9.2012, at 17.46, Damien Churchill wrote: >> >>> I was wondering what would be entailed in modifying dovecot 2.2 to >>> support storing mail in an object store. I've seen a few mails dotted >>> around in the ML history about supporting such a thing and seen it's >>> basically dependant upon some changes in lib-storage to support >>> writing messages without locking. Is this still the case? >> >> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >> > > That's good to hear :) I've been evaluating RADOS as an object store, > which is similar to S3. Although any distributed storage would be > great. I'd be more than happy to help code it! I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > > I'm assuming that the director would be used in order to distribute > connections to the same server, so it's only within a local instance > of dovecot you'd need to be aware of what currently has a connection > open for that user? Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > How are you planning on handling the situation where say node X dies > and hasn't uploaded the latest index file? Would that result in > missing messages from the mailbox when accessed by another node, or is > the local index intended to be more of a write-through cache? No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) From marcin at rootnode.net Mon Sep 17 15:57:31 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 14:57:31 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <8b54646de4ef445102e8f40ee8115c83@rootnode.net> On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? I have a service running with different UID that must have full access to mail folders. UID/user is added to vmail group. That is why 0770 permission is desired. -- M. From marcin at rootnode.net Mon Sep 17 16:25:27 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:25:27 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway > describes how it all works. Already tested and it does not work. Directory structure is /home/mail/// drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support Two last directories for mailbox support at maildomain.com are created automatically after first e-mail comes to this address. As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). Maybe additional configuration is needed to make it work? -- M. From tss at iki.fi Mon Sep 17 16:30:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 16:30:59 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> On 17.9.2012, at 16.25, Marcin H?ybin wrote: > On 2012-09-17 14:46, Timo Sirainen wrote: >> Yeah, although since they're already sharing UIDs I don't see much >> point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway >> describes how it all works. > > Already tested and it does not work. > Directory structure is /home/mail/// > > drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home > drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail > drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support > > Two last directories for mailbox support at maildomain.com are created automatically > after first e-mail comes to this address. > > As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). > Maybe additional configuration is needed to make it work? What's your mail_location setting? Note the "Permissions to new user directories" in the wiki, especially the last paragraph. When Dovecot doesn't know what permissions it should use, it defaults to 0700 to be safe. From CMarcus at Media-Brokers.com Mon Sep 17 16:40:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 09:40:09 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> Message-ID: <50572839.9000301@Media-Brokers.com> On 2012-09-17 8:45 AM, Timo Sirainen wrote: > Both CLucene and Lucene++ are C++ ports of the original Java Lucene. Hmmm... ok, I guess I'm missing something... Googling lucene++ leads me to this page: http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: http://lucene.apache.org/ and on that page it says (among other things): "/Lucene Core /, our flagship sub-project, provides Java-based indexing and search technology, as well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities." Which seems to say that it is java based...? -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 16:40:35 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:40:35 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 2012-09-17 15:30, Timo Sirainen wrote: > What's your mail_location setting? Note the "Permissions to new user > directories" in the wiki, especially the last paragraph. When Dovecot > doesn't know what permissions it should use, it defaults to 0700 to > be > safe. mail_location = maildir:~/Maildir Maybe that's the point. What do you propose? I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" -- M. From CMarcus at Media-Brokers.com Mon Sep 17 17:37:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:37:19 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50572839.9000301@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <5057359F.8070107@Media-Brokers.com> Ok, just to wrap up this thread for anyone who cares, I just got a reply from the clucene dev (in response to my asking him about the possible merger of clucene and lucene++) after he said he had fixed this bug and the next version would contain the fix: "More or less it's true. About a year ago we started to make Lucene++ to the new CLucene version, as Lucene++ (also written in C++) is a port of a newer Apache Lucene version (written in Java) as the one CLucene is a port of. But we did not want to simply merge them, but to adapt Lucene++ to the "design principles" of CLucene. E.g., Lucene++ makes heavy use of shared pointers. And in CLucene we wanted to reduce this usage in favor of performance. But this not finished and I cannot say when it will finished. Nevertheless, the new version of CLucene (if any) will be also C++ and not Java. Best regards, Veit" On 2012-09-17 9:40 AM, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache > Lucene/Solr page: > > http://lucene.apache.org/ > > and on that page it says (among other things): > > "/Lucene Core /, our flagship > sub-project, provides Java-based indexing and search technology, as > well as spellchecking, hit highlighting and advanced > analysis/tokenization capabilities." > > Which seems to say that it is java based...? > -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From tomas at blueboard.cz Mon Sep 17 17:40:55 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Mon, 17 Sep 2012 16:40:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes Message-ID: <50573677.7060108@blueboard.cz> Hello, I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 I`d like to have more secure configuration > one process to one connection. Thanks for help. Tomas My config: # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 6200 default_process_limit = 4096 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /var/log/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 1536 service_count = 1 } service imap { process_limit = 2048 } service pop3-login { service_count = 1 } service pop3 { process_limit = 2048 } shutdown_clients = no ssl_cert = References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: On 17.9.2012, at 16.40, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: > > http://lucene.apache.org/ This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus From tss at iki.fi Mon Sep 17 17:46:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:46:46 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573677.7060108@blueboard.cz> References: <50573677.7060108@blueboard.cz> Message-ID: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> On 17.9.2012, at 17.40, Tom?? Randa wrote: > Hello, > > I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). > Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. From tss at iki.fi Mon Sep 17 17:47:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:47:50 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 17.9.2012, at 16.40, Marcin H?ybin wrote: > On 2012-09-17 15:30, Timo Sirainen wrote: >> What's your mail_location setting? Note the "Permissions to new user >> directories" in the wiki, especially the last paragraph. When Dovecot >> doesn't know what permissions it should use, it defaults to 0700 to be >> safe. > > mail_location = maildir:~/Maildir > > Maybe that's the point. What do you propose? > I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" Then your current possibilities are: a) mkdir + chmod the directory when a new user is created b) change Dovecot sources From CMarcus at Media-Brokers.com Mon Sep 17 17:52:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:52:36 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <50573934.1070909@Media-Brokers.com> On 2012-09-17 10:44 AM, Timo Sirainen wrote: > This is the only page I'm aware of for the project: > https://github.com/luceneplusplus/LucenePlusPlus Weird... wonder which is which, or if they are one and the same... Not much activity on that one either (most things are over a year old)... Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? -- Best regards, Charles From support at oeko.net Mon Sep 17 18:08:53 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 17:08:53 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota Message-ID: <20120917150852.GA13113@spruce.wiehl.oeko.net> Hi, I have now verified that there is a problem with quota (and sieve?). Summary: If the "maildirsize" file gets garbled (= every once in a while), mail is getting lost randomly. Details: I have a Debian box (6.0.5/amd64) with a Postfix server (2.7.1), which delivers email via Dovecot (1.2.15) to local disk (ext3). My email is filtered via a sieve script (wish I could have something better, eg. procmail). The account data is fetched from an SQL database, and I have a default quota set in dovecot.conf (200MB). My own mailbox has no quota (*:storage=0, mentioned in the database). A client sent an email to me, which is logged as being delivered like normal to my mailbox, but doesn't show up there. After poking around, I decided to simply delete the maildirsize file and have the client send that email again, and voila, it shows up in my mailbox. At the same time, however, other emails get delivered just fine, so the heinous thing is that you don't immediately notice that something fishy is going on - only when the client actually calls you (sort of defeats some uses of email)... The problem is worsened by the fact that you can't tell *which* emails get lost - you only end up with a random subset of "your" messages. The problem occurred now after running for a few months. FWIW, the problem with the garbled maildirsize file resulted in a different user's email getting bounced due to "mailbox full", when it wasn't. Questions: * Is this a known problem (Google was unhelpful/unwieldy)? * How can I prevent the problem, short of setting up a cron job to delete that file every week? * How can I log the delivery of messages in dovecot's log, not only the POP3/IMAP activity when people fetch their email? TIA! Kind regards, --Toni++ From tomas at blueboard.cz Mon Sep 17 18:10:55 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:10:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> Message-ID: <50573D7F.1040005@blueboard.cz> Hello, thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. Tomas R. Dne 17.09.12 16:46, Timo Sirainen napsal(a): > On 17.9.2012, at 17.40, Tom?? Randa wrote: > >> Hello, >> >> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 > That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. > From tss at iki.fi Mon Sep 17 18:16:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:16:20 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. On 17.9.2012, at 18.10, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >> > From tss at iki.fi Mon Sep 17 18:18:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:18:37 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50573934.1070909@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> <50573934.1070909@Media-Brokers.com> Message-ID: On 17.9.2012, at 17.52, Charles Marcus wrote: > On 2012-09-17 10:44 AM, Timo Sirainen wrote: >> This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus > > Weird... wonder which is which, or if they are one and the same... > > Not much activity on that one either (most things are over a year old)... > > Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? Some very large installations are successfully using it. I'm not planning on breaking it anytime soon. From f.bonnet at esiee.fr Mon Sep 17 18:21:05 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:21:05 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50573FE1.10906@esiee.fr> hello what is the result of the following command ? sysctl -a | grep kern.maxproc On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From f.bonnet at esiee.fr Mon Sep 17 18:22:16 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:22:16 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50574028.3070608@esiee.fr> and this one sysctl -a | grep kern.maxprocperuid On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From tomas at blueboard.cz Mon Sep 17 18:24:24 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:24:24 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <505740A8.3090803@blueboard.cz> On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) Thanks for your time Tomas R. Dne 17.09.12 17:16, Timo Sirainen napsal(a): > Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. > > On 17.9.2012, at 18.10, Tom?? Randa wrote: > >> Hello, >> >> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >> >> >> Tomas R. >> >> >> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>> >>>> Hello, >>>> >>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>> From tss at iki.fi Mon Sep 17 18:26:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:26:10 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <505740A8.3090803@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> Message-ID: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. On 17.9.2012, at 18.24, Tom?? Randa wrote: > On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: > > cputime unlimited > filesize unlimited > datasize 33554432 kbytes > stacksize 524288 kbytes > coredumpsize unlimited > memoryuse unlimited > vmemoryuse unlimited > descriptors 36864 > memorylocked unlimited > maxproc 8192 > sbsize unlimited > swapsize unlimited > > > Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) > > > Thanks for your time > > Tomas R. > > > Dne 17.09.12 17:16, Timo Sirainen napsal(a): >> Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. >> >> On 17.9.2012, at 18.10, Tom?? Randa wrote: >> >>> Hello, >>> >>> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >>> >>> >>> Tomas R. >>> >>> >>> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>>> >>>>> Hello, >>>>> >>>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>>> > From tss at iki.fi Mon Sep 17 18:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:45:05 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> Message-ID: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> On 17.9.2012, at 18.26, Timo Sirainen wrote: > It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: 1. dovecot master process starts, inheriting the ulimits from the shell that starts it 2. Dovecot checks the current limit and increases it if necessary 3. Forked child processes inherit the parent's ulimits 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. kern.maxprocperuid I guess means the upper allowed limit for the ulimit. From tomas at blueboard.cz Mon Sep 17 18:58:12 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:58:12 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> Message-ID: <50574894.4090609@blueboard.cz> kern.maxprocperuid is maximum allowed user limit, But in standard it is set to maximum imho. And if dovecot checks limit why it did not increases it on FreeBSD? :( ? Hard limit 8192 on my box is much more. Dne 17.09.12 17:45, Timo Sirainen napsal(a): > On 17.9.2012, at 18.26, Timo Sirainen wrote: > >> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. > > Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: > > 1. dovecot master process starts, inheriting the ulimits from the shell that starts it > 2. Dovecot checks the current limit and increases it if necessary > 3. Forked child processes inherit the parent's ulimits > 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) > > There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. > > kern.maxprocperuid I guess means the upper allowed limit for the ulimit. > From support at oeko.net Mon Sep 17 19:06:40 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 18:06:40 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota In-Reply-To: <20120917150852.GA13113@spruce.wiehl.oeko.net> References: <20120917150852.GA13113@spruce.wiehl.oeko.net> Message-ID: <20120917160640.GA16928@spruce.wiehl.oeko.net> Hi, On Mon, Sep 17, 2012 at 05:08:53PM +0200, Toni Mueller wrote: > I have now verified that there is a problem with quota (and sieve?). I was probably jumping to conclusions, as the delivery problem happens again - statistically extremely unlikely after this short while. kind regards, --Toni++ From kgc at corp.sonic.net Tue Sep 18 02:02:29 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 17 Sep 2012 16:02:29 -0700 Subject: [Dovecot] Proxy connection timeouts Message-ID: <5057AC05.2050904@corp.sonic.net> We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) I'm curious if anyone else has seen any similar problems or has any suggestions. # dovecot -n # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Scientific Linux release 6.3 (Carbon) auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes auth_verbose_passwords = sha1 auth_worker_max_count = 64 mail_fsync = always mail_log_prefix = "%s(%u): session=%{session} " mail_plugins = stats zlib maildir_very_dirty_syncs = yes mmap_disable = yes passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = imap driver = pam } plugin { lazy_expunge = DELETED_MESSAGES. stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 service anvil { client_limit = 10000 } service auth { client_limit = 10000 vsz_limit = 512 M } service doveadm { inet_listener { port = 1842 } unix_listener doveadm-server { mode = 0666 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 7000 process_min_avail = 32 } service imap-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-imap-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 4096 vsz_limit = 512 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 2000 process_min_avail = 32 } service pop3-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-pop3-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } shutdown_clients = no ssl = required ssl_ca = References: <50449193.8080101@um.es> Message-ID: <50581BCC.7050607@um.es> El 11/09/12 16:24, Timo Sirainen escribi?: > On 3.9.2012, at 14.16, Angel L. Mateo wrote: > >> Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): >> >> doveadm auth cache flush >> 4 cache entries flushed > > Yep. > >> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) > > Works in my tests. > Is this cache the same than the user information cache? The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: doveadm auth cache flush in the logs I get: Sep 18 08:47:13 myotis34 dovecot: auth: Debug: master in: CACHE-FLUSH#0111#011 now I ask for user information with: doveadm user and this is what I get from logs: Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:19 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:19 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:41 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:41 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 As you can see in the third message, it is still using information from userdb cache From alessio at skye.it Tue Sep 18 10:50:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 18 Sep 2012 09:50:30 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> References: <5056CC95.8030209@skye.it> <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> Message-ID: <505827C6.6080204@skye.it> Il 17/09/2012 14:44, Timo Sirainen ha scritto: >> So, why don't provide a way to restrict shared mailboxes also for dict in SQL? >> >> One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. > If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. Good, after change my shared namespace from namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } to namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ ## <<=- change %u to %n separator = / subscriptions = no type = shared } shared mailboxes have become available only inside the same domains. Thanks! -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From ap00 at mail.ru Tue Sep 18 14:04:31 2012 From: ap00 at mail.ru (=?UTF-8?B?0JDQu9C10LrRgdC10Lkg0J/QsNC90LrRgNCw0YLQvtCy?=) Date: Tue, 18 Sep 2012 15:04:31 +0400 Subject: [Dovecot] =?utf-8?q?lda_bug=3A_multiple_passdb_ignored?= Message-ID: <1347966271.760993075@f26.mail.ru> Greetings. When dovecot configured to use multiple passdb and static userdb as: #password database in multiple ldap subtree passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-real.conf } passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-virtual.conf } userdb { ?driver = static ?args = uid=pop gid=mailnull } all works fine but... When LDA deliver mail it check passdb but only first appeared in configuration: 18.09.2012 15:00:18 auth: Debug: master in: USER??????? 1?????? info at zzz.zz ???? service=lda 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword 18.09.2012 15:00:18 lda: Debug: auth input: 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND?? 1 1 P.S. dovecot version 2.1.7 sysytem x86 FreeBSD 9.0 From janfrode at tanso.net Tue Sep 18 14:38:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 18 Sep 2012 13:38:53 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? Message-ID: We have a sieve script doing sieve_before to sort spam to spam-folders. Now I'm trying to configure the "Trash" plugin, but it doesn't seem to work.. I noticed my config file says: # Space separated list of plugins to load (none known to be useful so far). Do NOT # try to load IMAP plugins here. #mail_plugins = and that doveconf doesn't list any plugins loaded for "protocol sieve". Should we load quota and trash here ? -jf From Jost.Krieger+dovecot at rub.de Tue Sep 18 14:40:12 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 18 Sep 2012 13:40:12 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) Message-ID: <20120918114011.GH8536@ruhr-uni-bochum.de> It seems there is no way to correct te quota limit downward for some mailboxes, like: quota_rule9 = UCE-TMP:storage=-10%% The idea is to not eat up the users storage with spam, so that spam stays in the queue (for some time) while real mail gets delivered. There *is* a line } else if (*value != '-' && relative_rule) { in quota.c, but nothing else handles it, starting with quota_rule_parse_percentage. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From stephan at rename-it.nl Tue Sep 18 16:41:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 15:41:16 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? In-Reply-To: References: Message-ID: <505879FC.4010907@rename-it.nl> On 9/18/2012 1:38 PM, Jan-Frode Myklebust wrote: > We have a sieve script doing sieve_before to sort spam to > spam-folders. Now I'm trying to configure the "Trash" plugin, but it > doesn't seem to work.. I noticed my config file says: > > # Space separated list of plugins to load (none known to be useful > so far). Do NOT > # try to load IMAP plugins here. > #mail_plugins = > > and that doveconf doesn't list any plugins loaded for "protocol > sieve". Should we load quota and trash here ? Keep in mind that the Sieve language and the Sieve protocol are two different things. The Sieve language is what processes incoming e-mails; for Dovecot it is executed by a plugin for LTMP or LDA during message delivery. The Sieve protocol - which is actually called ManageSieve - is used to upload Sieve scripts to the Server. That mail_plugins setting applies to the Sieve protocol, for which currently no useful plugins are known. The Sieve protocol service does not (currently) open any mail stores, which makes configuring mail-related plugins for it - such as quota and trash - utterly useless. So, in short, the answer to the above question is no. To help you further, we need some more information about your configuration. First of all, please provide the output of dovecot -n to this list. Also, describe more thoroughly what is going wrong. Do you see any log messages, etc. Regards, Stephan. From stephan at rename-it.nl Tue Sep 18 17:34:51 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 16:34:51 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058868B.8080904@rename-it.nl> Hello Dovecot users, I've been busy recently with some wholly unrelated Dovecot IMAP patches, so Sieve development moved a bit into the background. This release therefore consists mostly of small bugfixes. The most important change is the fix of a bug in ManageSieve's CHECKSCRIPT command that causes a segfault. Quite a few people have encountered this problem recently, so it is about time I release this now. Changelog v0.3.2: + sieve-refilter tool: improved man page documentation by explicitly specifying the syntax used for mailbox arguments. + Sieve: spamtest and virustest extensions: improved trace debugging of score calculation. + Sieve: made error messages about exceeding the maximum number of actions more verbose. - Sieve tools: fixed problems with running as root: sievec and sieve-dump now ignore mail_uid and mail_gid settings when run as root. - Sieve: fixed bug in action accounting (for limit checking): increase action instance count only when an action is actually created. - Sieve: include extension: fixed namespace separation of :global and :personal scripts. - ManageSieve: fixed segfault bug triggered by CHECKSCRIPT command. - Fixed linking with ld.gold. - Fixed several Clang compile warnings and a few potential bugs. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Tue Sep 18 18:48:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 18:48:19 +0300 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: This is now implemented as a plugin in hg. Usage in the commit message: http://hg.dovecot.org/dovecot-2.1/rev/f5bb9f6b304d Also there's a generic change where symlinks within same directory are treated as mailbox aliases, which aren't counted towards quota. On 2.9.2012, at 18.32, Timo Sirainen wrote: > I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. > > So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: > > .. > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > .. > > If "Sent Messages" doesn't exist: > - DELETE Sent works as usual > - RENAME Sent works as usual > > If "Sent Messages" is created: > - it's symlinked to Sent > - DELETE "Sent messages" deletes the symlink, not the contents > - RENAME "Sent messages" isn't allowed > - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. > > Thoughts? > > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Tue Sep 18 19:04:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:04:06 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > passdb { > driver = ldap > args = /usr/local/etc/dovecot/dv-ldap-virtual.conf > } Does dv-ldap-virtual.conf have auth_bind=yes? From tss at iki.fi Tue Sep 18 19:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:17:50 +0300 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120918114011.GH8536@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: On 18.9.2012, at 14.40, Jost Krieger wrote: > It seems there is no way to correct te quota limit downward for some > mailboxes, like: > > quota_rule9 = UCE-TMP:storage=-10%% > > The idea is to not eat up the users storage with spam, so that spam > stays in the queue (for some time) while real mail gets delivered. Interesting idea. This was pretty easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. From tss at iki.fi Tue Sep 18 19:31:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:31:08 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50581BCC.7050607@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> Message-ID: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >> >> Works in my tests. >> > Is this cache the same than the user information cache? Yes. > The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: > > doveadm auth cache flush What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: doveadm auth cache flush foo # make sure it isn't there doveadm user foo doveadm auth cache flush foo Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. From tss at iki.fi Tue Sep 18 19:34:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:34:49 +0300 Subject: [Dovecot] Proxy connection timeouts In-Reply-To: <5057AC05.2050904@corp.sonic.net> References: <5057AC05.2050904@corp.sonic.net> Message-ID: <15C8389A-3C3A-461B-8FF8-FBACBDDB6A6C@iki.fi> On 18.9.2012, at 2.02, Kelsey Cummings wrote: > We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. > > dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) > > I'm curious if anyone else has seen any similar problems or has any suggestions. I once had similar problems when the proxy backend was Courier. The problems went away after migration to Dovecot was complete. The possibilities are either: a) The backend server is busy and doesn't have a chance to accept() the connection. b) Packets get dropped in the network and the retry packet is slow in coming (or also gets lost). Changing some kernel settings might help with a). There are also kernel settings that specify how SYN resend is attempted, you could try reducing it to a few seconds. From ap00 at mail.ru Tue Sep 18 19:40:39 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 20:40:39 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> References: <1347966271.760993075@f26.mail.ru> <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> Message-ID: <106-722288046.20120918204039@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:04:06 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> passdb { >> driver = ldap >> args = /usr/local/etc/dovecot/dv-ldap-virtual.conf >> } TS> Does dv-ldap-virtual.conf have auth_bind=yes? No. All ldap connection configured to use one prescribed ldap binding. P.S. Thank you for your work. -- Best regards, Anthony mailto:ap00 at mail.ru From tss at iki.fi Tue Sep 18 19:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:58:53 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > When dovecot configured to use multiple passdb and static userdb as: .. > all works fine but... > > When LDA deliver mail it check passdb but only first appeared in configuration: > > 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda > 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss > 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword > 18.09.2012 15:00:18 lda: Debug: auth input: > 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: > 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 From robert at schetterer.org Tue Sep 18 20:47:09 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 18 Sep 2012 19:47:09 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <5058B39D.5030501@schetterer.org> Am 18.09.2012 18:17, schrieb Timo Sirainen: > On 18.9.2012, at 14.40, Jost Krieger wrote: > >> It seems there is no way to correct te quota limit downward for some >> mailboxes, like: >> >> quota_rule9 = UCE-TMP:storage=-10%% >> >> The idea is to not eat up the users storage with spam, so that spam >> stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf > > Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. > some allready existing way is i.e with milter and postfix tagged spam get in hold queue for i.e human admin investigate, or some quarantaine with amavis for sure there are benefits getting dovecot involved in spam tagging but i would leave spam tagging to the mailserver in first way and deliver it to Junk folder by global sieve rule with giving more quota or ignoring Quota to Junk folder, but i see this isnt the same what you might wanna goal -- Best Regards MfG Robert Schetterer From jcblanco at fi.upm.es Tue Sep 18 21:18:21 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 18 Sep 2012 20:18:21 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058BAED.5080505@fi.upm.es> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the new pigeonhole version, however I've got the follwing errors when compiling: gcc -shared .libs/rfc2822.o .libs/edit-mail.o .libs/sieve-settings.o .libs/sieve-message.o .libs/sieve-smtp.o .libs/sieve-lexer.o .libs/sieve-script.o .libs/sieve-script-file.o .libs/sieve-script-dict.o .libs/sieve-ast.o .libs/sieve-binary.o .libs/sieve-binary-file.o .libs/sieve-binary-code.o .libs/sieve-binary-debug.o .libs/sieve-parser.o .libs/sieve-address.o .libs/sieve-validator.o .libs/sieve-generator.o .libs/sieve-interpreter.o .libs/sieve-runtime-trace.o .libs/sieve-code-dumper.o .libs/sieve-binary-dumper.o .libs/sieve-result.o .libs/sieve-error.o .libs/sieve-objects.o .libs/sieve-stringlist.o .libs/sieve-comparators.o .libs/sieve-match-types.o .libs/sieve-address-parts.o .libs/sieve-match.o .libs/sieve-commands.o .libs/sieve-code.o .libs/sieve-actions.o .libs/sieve-extensions.o .libs/sieve-plugins.o .libs/cmp-i-octet.o .libs/cmp-i-ascii-casemap.o .libs/mcht-is.o .libs/mcht-contains.o .libs/mcht-matches.o .libs/tst-truefalse.o .libs/tst-not.o .libs/tst-anyof.o .libs/tst-allof.o .libs/tst-address.o .libs/tst-header.o .libs/tst-exists.o .libs/tst-size.o .libs/cmd-require.o .libs/cmd-stop.o .libs/cmd-if.o .libs/cmd-keep.o .libs/cmd-redirect.o .libs/cmd-discard.o .libs/ext-fileinto.o .libs/ext-reject.o .libs/ext-envelope.o .libs/ext-encoded-character.o .libs/sieve.o -Wl,--whole-archive ../../src/lib-sieve/plugins/vacation/.libs/libsieve_ext_vacation.a ../../src/lib-sieve/plugins/subaddress/.libs/libsieve_ext_subaddress.a ../../src/lib-sieve/plugins/comparator-i-ascii-numeric/.libs/libsieve_ext_comparator-i-ascii-numeric.a ../../src/lib-sieve/plugins/relational/.libs/libsieve_ext_relational.a ../../src/lib-sieve/plugins/regex/.libs/libsieve_ext_regex.a ../../src/lib-sieve/plugins/copy/.libs/libsieve_ext_copy.a ../../src/lib-sieve/plugins/imap4flags/.libs/libsieve_ext_imap4flags.a ../../src/lib-sieve/plugins/include/.libs/libsieve_ext_include.a ../../src/lib-sieve/plugins/body/.libs/libsieve_ext_body.a ../../src/lib-sieve/plugins/variables/.libs/libsieve_ext_variables.a ../../src/lib-sieve/plugins/enotify/.libs/libsieve_ext_enotify.a ../../src/lib-sieve/plugins/notify/.libs/libsieve_ext_notify.a ../../src/lib-sieve/plugins/environment/.libs/libsieve_ext_environment.a ../../src/lib-sieve/plugins/mailbox/.libs/libsieve_ext_mailbox.a ../../src/lib-sieve/plugins/date/.libs/libsieve_ext_date.a ../../src/lib-sieve/plugins/spamvirustest/.libs/libsieve_ext_spamvirustest.a ../../src/lib-sieve/plugins/ihave/.libs/libsieve_ext_ihave.a ../../src/lib-sieve/plugins/editheader/.libs/libsieve_ext_editheader.a ../../src/lib-sieve/plugins/vnd.dovecot/debug/.libs/libsieve_ext_debug.a ../../src/lib-sieve/plugins/vnd.dovecot/duplicate/.libs/libsieve_ext_duplicate.a -Wl,--no-whole-archive -Wl,--rpath -Wl,/usr/lib64/dovecot -L/var/tmp/dovecot-2.1-pigeonhole-0.3.2-219_3-root/usr/lib64/dovecot -L/usr/lib64/dovecot -ldovecot-lda -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-storage/.libs -L/usr/kerberos/lib64 -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-dovecot/.libs -lrt -m64 -mtune=generic -Wl,-soname -Wl,libdovecot-sieve.so.0 -o .libs/libdovecot-sieve.so.0.0.0 /usr/bin/ld: cannot find -ldovecot-lda collect2: ld returned 1 exit status libtool: install: error: relink `libdovecot-sieve.la' with the above command before installing it make[4]: *** [install-dovecot_pkglibLTLIBRARIES] Error 1 make[4]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src' make: *** [install-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.50565 (%install) It seems that the error is related to the ld.gold fix, If I revert the c52a0c561311 patch the error is not produced Regards From stephan at rename-it.nl Tue Sep 18 21:49:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 20:49:19 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058BAED.5080505@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> Message-ID: <5058C22F.1070704@rename-it.nl> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: > Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the > new pigeonhole version, however I've got the follwing errors when > compiling: > > It seems that the error is related to the ld.gold fix, If I revert the > c52a0c561311 patch the error is not produced Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, but I forgot the Dovecot v2.1 tree. Fixed now: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c This only happens when you compile against installed Dovecot headers, people using --with-dovecot= (like me) would not have seen this. I'll push out a new release once 2.1.10 comes out. I'm being told that is imminent. Regards, Stephan. From tss at iki.fi Tue Sep 18 21:52:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 21:52:27 +0300 Subject: [Dovecot] v2.1.10 released Message-ID: <1347994347.13571.67.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig The director bugfix is pretty important at least when running in two director setups, since it can break the directors until both of them are manually shut down. I'm not entirely sure if the same happens in >2 director setups, but I'd think so (although I do wonder why I haven't seen it happen in them). BTW. I'm hoping for v2.2.alpha1 release within a week or two and to get it stabilized rather quickly. + imap: Implemented THREAD=ORDEREDSUBJECT extension. + Added "doveadm exec" command to easily execute commands from libexec_dir, e.g. "doveadm exec imap -u user at domain" + Added "doveadm copy" command. + doveadm copy/move: Added optional user parameter to specify the source username. This allows easily copying mails between different users. + Added namespace { disabled } setting to quickly enable/disable namespaces. This is especially useful when its value is returned by userdb. + Added mailbox_alias plugin. It allows creating mailbox aliases using symlinks. + imapc storage: Added imapc_max_idle_time setting to force activity on connection. + fts-solr: Expunging multiple messages is now faster. - director: In some conditions director may have disconnected from another director (without logging about it), thinking it was sending invalid data. - imap: Various fixes to listing mailboxes. - pop3-migration plugin: Avoid disconnection from POP3 server due to idling. - login processes crashed if there were a lot of local {} or remote {} settings blocks. From ap00 at mail.ru Tue Sep 18 22:04:42 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 23:04:42 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> References: <1347966271.760993075@f26.mail.ru> <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> Message-ID: <186-713645281.20120918230442@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:58:53 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> When dovecot configured to use multiple passdb and static userdb as: TS> .. >> all works fine but... >> >> When LDA deliver mail it check passdb but only first appeared in configuration: >> >> 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda >> 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss >> 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword >> 18.09.2012 15:00:18 lda: Debug: auth input: >> 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: >> 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 TS> This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 Thank you very much. -- Best regards, Anthony mailto:ap00 at mail.ru From stephan at rename-it.nl Tue Sep 18 22:36:34 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 21:36:34 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.3 for Dovecot v2.1.10 Message-ID: <5058CD42.3000507@rename-it.nl> Hello Dovecot users, Version 0.3.2 was a bit broken, because it failed to compile against installed Dovecot headers. Changelog v0.3.3: - Fixed compile against installed Dovecot headers. This was broken by the ld.gold fix in the previous release. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From interfasys at gmail.com Tue Sep 18 23:32:18 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 21:32:18 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) Message-ID: <5058DA52.7070400@gmail.com> Hello, There is a big problem with 2.1.10. Here is what I saw in the logs: Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed with signal 6 (core dumped) Reverting to 2.1.9 with the exact same config file works just fine Cheers, Olivier From Ralf.Hildebrandt at charite.de Tue Sep 18 23:34:03 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 18 Sep 2012 22:34:03 +0200 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <20120918203403.GK15878@charite.de> * interfaSys s?rl : > Hello, > > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine So far I'm not having problems wit 2.1.10 here (coming from 2.1.9) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 18 23:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 23:42:57 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> On 18.9.2012, at 23.32, interfaSys s?rl wrote: > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine gdb backtrace? http://dovecot.org/bugreport.html From interfasys at gmail.com Wed Sep 19 00:18:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 22:18:07 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> Message-ID: <5058E50F.10000@gmail.com> I thought it would be easily identifiable and a common problem, but apparently not :) Here is the trace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at user.c:17 _module_tmp = (void *) 0x10a84d28 asu = (struct antispam_user *) 0x10a84d28 tmp = 0x0 #9 0x00000000104f1f04 in hook_mail_user_created () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #10 0x00000000104efaf9 in mail_user_init () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000010495858 in mail_storage_service_init_post.46586.6405 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000010495582 in mail_storage_service_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #13 0x00000000104951b3 in mail_storage_service_lookup_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #14 0x000000000041f112 in client_create_from_input (input=0x7fffffffe940, login_client=0x10a550c0, fd_in=11, fd_out=11, input_buf=0x7fffffffe900, error_r=0x7fffffffe938) at main.c:202 user = (struct mail_storage_service_user *) 0x10a55141 mail_user = (struct mail_user *) 0x3 client = (struct client *) 0x7fffffffec70 set = (const struct imap_settings *) 0x7fffffffe900 flags = 32767 #15 0x000000000041eeaa in login_client_connected (client=0x10a550c0, username=0x10a04093 "test at example.com", extra_fields=0x10a04118) at main.c:272 input = {module = 0x423d0f "imap", service = 0x423d0f "imap", username = 0x10a04093 "test at example.com", session_id = 0x10a55120 "UCFMXQDK5gBewxZf", local_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "N.8Q", '\0' , __u6_addr16 = {11854, 20792, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1362636366, 0, 0, 0}}}, ip4 = {s_addr = 1362636366}}}, remote_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "^?\026_", '\0' , __u6_addr16 = {50014, 24342, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1595327326, 0, 0, 0}}}, ip4 = {s_addr = 1595327326}}}, local_port = 0, remote_port = 0, userdb_fields = 0x10a04118, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} error = (const __unknown__ *) 0x0 input_buf = {data = 0x10a55141, used = 3, priv = {0x0, 0x0, 0x3, 0x0, 0x0}} #16 0x00000000105a229f in master_login_auth_finish.11302 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x00000000105efa50 in master_login_auth_callback.11253.2275 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #18 0x0000000010596f56 in master_login_auth_input_user.11520 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x000000001058bf99 in master_login_auth_input.11492 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00000000105e9b34 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x00000000105d1ba7 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #22 0x00000000105d1a55 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x00000000105d1790 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #24 0x000000000041ed5d in main (argc=1, argv=0x7fffffffec60) at main.c:389 set_roots = {0x424020, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ed9d , failure_callback = 0x4194ee } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x424020, 0x0} Cheers, Olivier On 18/09/2012 21:42, Timo Sirainen wrote: > On 18.9.2012, at 23.32, interfaSys s?rl wrote: > >> There is a big problem with 2.1.10. >> >> Here is what I saw in the logs: >> Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 >> (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) >> Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed >> with signal 6 (core dumped) >> >> Reverting to 2.1.9 with the exact same config file works just fine > > gdb backtrace? http://dovecot.org/bugreport.html > > > From tss at iki.fi Wed Sep 19 02:42:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 02:42:34 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <1CB6DE70-A784-4D90-8127-BF87EDA84407@iki.fi> On 19.9.2012, at 0.18, interfaSys s?rl wrote: > I thought it would be easily identifiable and a common problem, but > apparently not :) .. > #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at > user.c:17 Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. From robert at schetterer.org Wed Sep 19 10:40:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 09:40:29 +0200 Subject: [Dovecot] v2.1.10 released In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <505976ED.4090602@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > + Added mailbox_alias plugin. It allows creating mailbox aliases using > symlinks. Hi Timo thx for coding, no Problems so far, after update, but mailbox_alias plugin should be described at http://wiki2.dovecot.org/Plugins/ -- Best Regards MfG Robert Schetterer From bangkokmaco at gmail.com Wed Sep 19 11:00:55 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 19 Sep 2012 15:00:55 +0700 Subject: [Dovecot] ssl cert for mail server Message-ID: for testing a new ssl cert. it works ok for browsers, but > openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 => verify error:num=19:self signed certificate in certificate chain is this ssl cert - as it's constructed - is ok for mail clients? (realize needs to be on mail port etc - right now talking about the cert itself.) have had problems with thunderbird, and was wondering if this might be part of the problem. From amateo at um.es Wed Sep 19 11:22:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 10:22:42 +0200 Subject: [Dovecot] Deleting a folder with & character Message-ID: <505980D2.3060401@um.es> Hello, One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). I can see the maildir with a doveadm list command: amateo_adm at myotis31:~$ sudo doveadm mailbox list -u ... INBOX.Kron & SPM ... but I can't delete it, neithe rename it: amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? From acrow at integrafin.co.uk Wed Sep 19 11:29:12 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Wed, 19 Sep 2012 09:29:12 +0100 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <50598258.1030005@integrafin.co.uk> On 19/09/12 09:22, Angel L. Mateo wrote: > Hello, > > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe > created a long time ago, when we use courier imap as pop/imap server, > but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox > doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox > doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a > specific syntax or escape to indicate the name? > Hi, I see something similar when I try to doveadm import maildir folders (to mdbox) with ampersands - I get the same "Mailbox doesn't exist: INBOX.Foo &- Bar" with that extraneous hyphen. Cheers Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. "Transact" is operated by Integrated Financial Arrangements plc Domain House, 5-7 Singer Street, London EC2A 4BQ Tel: (020) 7608 4900 Fax: (020) 7608 5300 (Registered office: as above; Registered in England and Wales under number: 3727592) Authorised and regulated by the Financial Services Authority (entered on the FSA Register; number: 190856) From tss at iki.fi Wed Sep 19 12:14:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 12:14:07 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> On 19.9.2012, at 11.22, Angel L. Mateo wrote: > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). From amateo at um.es Wed Sep 19 12:15:53 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 11:15:53 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <50598D49.8060507@um.es> El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > I have made the test in my test server (it has no real activity). In this server, user entry is refreshed correctly. But the same test in my production servers fails. I have checked (in the production one) that the second flush delete entries (in fact, 2, not 1): amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M but quota information is not reloaded from ldap server. I have also checked my ldap server, and dovecot is not performing any search operation for the user after flushing him from the cache. I have attached my doveconf -n. In my ldap configuration I have: user_attrs = irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule pass_attrs = irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule and I have check that my test and production server has the same configuration (but client_limit and number of processes). Any idea? From c at roessner-network-solutions.com Wed Sep 19 12:26:33 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 11:26:33 +0200 Subject: [Dovecot] OT: z-Push Message-ID: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Hi, just a short question: I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. Also I can not remove that folder over ActiveSync. Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 Thanks in advance -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From robert at schetterer.org Wed Sep 19 12:42:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 11:42:12 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: <50599374.4010902@schetterer.org> Am 19.09.2012 11:26, schrieb Christian R??ner: > Hi, > > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? yes it works with dovecot > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 sorry tested , there is no option for create folder on my android, need test account offlist? > > Thanks in advance > > -Christian R??ner > > --- > Bachelor of Science Informatik > Erlenwiese 14, 36304 Alsfeld > T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 > USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com > -- Best Regards MfG Robert Schetterer From CMarcus at Media-Brokers.com Wed Sep 19 13:36:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 19 Sep 2012 06:36:12 -0400 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> Message-ID: <5059A01C.2080706@Media-Brokers.com> On 2012-09-19 5:14 AM, Timo Sirainen wrote: > It's an invalid mailbox name and you can't access it using doveadm. > Either rename it with "mv" or via IMAP protocol (not entirely sure if > it works via IMAP either). Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? Thanks Timo, -- Best regards, Charles From alessio at skye.it Wed Sep 19 13:54:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 12:54:33 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <5059A469.6060604@skye.it> Hi, I have found this strange problem. I'm working with Debian 6, dovecot 2.1.9 and vpopmail-auth. LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. If disable ACL (acl and imap_acl from mail_plugins) in dovecot.conf "Segmentation fault" go away. Moreover, without ACL at the first delivery folders like Sent, Drafts, Trash, Spam appear immediately. An example, new user: root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? maildirsize ??? new ??? tmp root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Segmentation fault root at demo-vpop ~ # echo $? 139 root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? dovecot-acl-list ??? dovecot.index.cache ??? dovecot.index.log ??? dovecot-uidlist ??? dovecot-uidvalidity ??? dovecot-uidvalidity.50599a7f ??? maildirsize ??? new ? ??? 1348049535.M21758P19264.demo-vpop.cbsolt.net,S=2025,W=2071 ??? tmp dovecot log (with ulimit -c unlimited before starting Dovecot): Sep 19 12:06:50 auth: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:06:50 auth-worker(18728): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth-worker(18728): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:06:50 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 dict: Info: mysql(localhost): Connected to database dovecot lda log: Sep 19 12:06:50 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:06:50 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found now works fine: root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Sep 19 12:08:18 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:08:18 auth-worker(18763): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:08:18 auth-worker(18763): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:08:18 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 dict: Info: mysql(localhost): Connected to database dovecot Sep 19 12:08:18 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:08:18 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M } protocols = imap pop3 sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service pop3-login { service_count = 0 } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From florob at babelmonkeys.de Wed Sep 19 14:56:26 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 13:56:26 +0200 Subject: [Dovecot] ssl cert for mail server In-Reply-To: References: Message-ID: <5059B2EA.3040605@babelmonkeys.de> Am 19.09.2012 10:00, schrieb cc "maco" young: > for testing a new ssl cert. it works ok for browsers, but > >> openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 > > => verify error:num=19:self signed certificate in certificate chain > > is this ssl cert - as it's constructed - is ok for mail clients? (realize > needs to be on mail port etc - right now talking about the cert itself.) > have had problems with thunderbird, and was wondering if this might be > part of the problem. > Hi, first of all this is likely off topic for this ML, I'll still answer though, since I'm always intrigued by TLS problems. The reason openssl doesn't accept this cert, while your browser does, is quite likely that your system wide accepted CAs don't include Starfield Technologies, while your browser's CAs do (This is the case for Firefox and Thunderbird). However, I suspect that your mail addresses are of the form , and ms1.trailsandtribulations.net is what is in your MX record. As such the certificate needs to be valid for trailsandtribulations.net, and not ms1.trailsandtribulations.net. So you either need trailsandtribulations.net as your CN, or a SAN of type DNSName for trailsandtribulations.net. Cf. https://tools.ietf.org/html/rfc6125 for best practices on generating certificates. Regards, Florian From ml at cibest.com Wed Sep 19 15:15:48 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 19 Sep 2012 14:15:48 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <50504ED3.5030601@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> Message-ID: <5059B774.6060808@cibest.com> On 12/09/2012 10:58, Cibest ML wrote: > On 11/09/2012 20:48, Timo Sirainen wrote: >> On 8.9.2012, at 18.07, Cibest ML wrote: >> >>> - Message delivered by LMTP >>> Quota name Type Value Limit >>> User quota STORAGE 2653 524288 >>> User quota MESSAGE 1 - >>> >>> - quota recalc >>> Quota name Type Value Limit >>> User quota STORAGE 5307 524288 >>> User quota MESSAGE 2 - >> Quota recalculation is clearly seeing the message twice. My first >> guess was that you had two namespaces pointing to the same location, >> but you seem to have only one. Do you have symlinks in your Maildir? >> > Hi, > Nothing specific on the namespace side. > The only entry is the default one in 10-mail.conf > namespace inbox { > inbox = yes > } > > I tried to comment it out but it doesn't change anything > > And no symlinks in the Maildir. > Here's the file structure of Maildir for the test mailbox > > ./dovecot-uidvalidity.504df4e7 > ./dovecot-uidvalidity > ./.Sent > ./.Sent/maildirfolder > ./.Sent/new > ./.Sent/dovecot.index.log > ./.Sent/tmp > ./.Sent/dovecot-uidlist > ./.Sent/cur > ./.Spam > ./.Spam/maildirfolder > ./.Spam/new > ./.Spam/dovecot.index.log > ./.Spam/tmp > ./.Spam/dovecot-uidlist > ./.Spam/cur > ./new > ./subscriptions > ./.Drafts > ./.Drafts/maildirfolder > ./.Drafts/new > ./.Drafts/dovecot.index.log > ./.Drafts/tmp > ./.Drafts/dovecot-uidlist > ./.Drafts/cur > ./.Drafts/dovecot-keywords > ./dovecot.index.log > ./tmp > ./dovecot.mailbox.log > ./dovecot-uidlist > ./.Trash > ./.Trash/maildirfolder > ./.Trash/new > ./.Trash/dovecot.index.log > ./.Trash/tmp > ./.Trash/dovecot-uidlist > ./.Trash/cur > ./cur > ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa > ./dovecot-keywords > ./dovecot.index.cache > > -- > Regards > S?bastien We did some more investigations as we had to enable the no enforcing quota option to prevent too many compltains from our customers. As the issue seems to be linked to the inbox folder, we tried to focus on that. According to Timo's last post, that could be linked to namespace or symlinks (we don't have any). There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). Now doing some check via telnet doesn't show any mysterious "Inbox" list "" "*" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Spam" * LIST (\HasNoChildren) "." "Drafts" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "INBOX" lsub "" "*" * LSUB () "." "Trash" * LSUB () "." "Sent" * LSUB () "." "Spam" * LSUB () "." "Drafts" * LSUB () "." "Inbox" 04 select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. 04 select Inbox * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. It's working for both but I checked on other working dovecot servers, that seems to be a normal behavior for Inbox folder I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) -- S?bastien From tss at iki.fi Wed Sep 19 15:42:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:42:16 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <5059A01C.2080706@Media-Brokers.com> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> Message-ID: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> On 19.9.2012, at 13.36, Charles Marcus wrote: > On 2012-09-19 5:14 AM, Timo Sirainen wrote: >> It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). > > Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. Dovecot has also some additional restrictions on characters (mainly control chars), but they are only enforced when creating new mailboxes. Opening existing mailboxes with control chars is allowed. From tss at iki.fi Wed Sep 19 15:47:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:47:28 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5059B774.6060808@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> Message-ID: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> On 19.9.2012, at 15.15, Cibest ML wrote: > There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). No idea. INBOX is treated case-insensitively by IMAP protocol. > I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. From tss at iki.fi Wed Sep 19 15:48:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:48:27 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059A469.6060604@skye.it> References: <5059A469.6060604@skye.it> Message-ID: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> On 19.9.2012, at 13.54, Alessio Cecchi wrote: > LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:52:41 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:52:41 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <20120919125241.GF2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:17:50 PM GMT, Timo Sirainen wrote: > > On 18.9.2012, at 14.40, Jost Krieger wrote: > ... > > The idea is to not eat up the users storage with spam, so that spam > > stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Thank you! > Now, the next problem is that if LDA can't save the message to UCE-TMP > because of quota failure it saves it to INBOX. I'm not really sure how > that should be fixed, since generally it's a good idea to do it.. Oops, that's not what I want! How about another command line parameter for lda to suppress the emergency save? We know exactly when we are trying to save probable spam. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:53:39 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:53:39 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5058B39D.5030501@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> Message-ID: <20120919125339.GG2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > some allready existing way is i.e > with milter and postfix tagged spam get in hold queue for i.e human > admin investigate, or some quarantaine with amavis Human admin is not a good idea if you have 50000 users :-) Quarantine's a possibility, but that's taking it out of "Mail" and adds complexity. > for sure there are benefits getting dovecot involved in spam tagging > but i would leave spam tagging to the mailserver in first way and > deliver it to Junk folder by global sieve rule with giving more quota or > ignoring Quota to Junk folder, but i see this isnt the same what you > might wanna goal That's just what we want to do, *but* although it's in the Wiki I don't think it a good idea to *raise* the limit for the junk folder if you *deliver* to it. It's somewhat ok if you move mail there and someone picks it up (we do that with a different folder). But on delivery it means at some point you can receive spam but no real messages. Not nice. "ignore" may work if it does what I think (never even count bytes and messages in this folder) but it is open to abuse ... Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From amateo at um.es Wed Sep 19 16:00:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 15:00:45 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50598D49.8060507@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <50598D49.8060507@um.es> Message-ID: <5059C1FD.1040702@um.es> (I forgot the attach) El 19/09/12 11:15, Angel L. Mateo escribi?: > El 18/09/12 18:31, Timo Sirainen escribi?: >> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >> >>>>> So I'm running this command. Whenever I run it, I get the >>>>> message that 3 (sometimes, is 4) entries are removed, but user >>>>> information isn't really reloaded and I doubt it is really removed >>>>> from cache (I have the user in a passwd-file and information used >>>>> by imap processes is still the old one, no the new one, changed >>>>> before the flush) >>>> >>>> Works in my tests. >>>> >>> Is this cache the same than the user information cache? >> >> Yes. >> >>> The parameter of the user I want to change is his quota, so I >>> have modified quota value in my ldap diretory, then I run: >>> >>> doveadm auth cache flush >> >> What is your doveconf -n output and the dovecot-ldap.conf contents? Is >> with or without @domain? Also try this: >> >> doveadm auth cache flush foo # make sure it isn't there >> doveadm user foo >> doveadm auth cache flush foo >> >> Does the second flush return 1 or 0 entries? If 0, then there's a >> problem. If 1, then it really should have worked. >> >> You could try also if disabling userdb prefetch makes any difference. >> And if you still have multiple userdb try with only one. >> > > I have made the test in my test server (it has no real activity). > In this server, user entry is refreshed correctly. But the same test in > my production servers fails. > > I have checked (in the production one) that the second flush delete > entries (in fact, 2, not 1): > > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > > but quota information is not reloaded from ldap server. > > I have also checked my ldap server, and dovecot is not performing > any search operation for the user after flushing him from the cache. > > I have attached my doveconf -n. In my ldap configuration I have: > user_attrs = > irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule > > pass_attrs = > irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule > > > and I have check that my test and production server has the same > configuration (but client_limit and number of processes). > > Any idea? > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Wed Sep 19 16:03:58 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:03:58 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> Message-ID: <5059C2BE.7050006@skye.it> Il 19/09/2012 14:48, Timo Sirainen ha scritto: > On 19.9.2012, at 13.54, Alessio Cecchi wrote: > >> LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. > gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html > Obvious ... :-) Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi10 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) is enough? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Wed Sep 19 16:07:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:07:31 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C2BE.7050006@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> Message-ID: <5059C393.5050209@skye.it> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: > Il 19/09/2012 14:48, Timo Sirainen ha scritto: >> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >> >>> LDA is configured and works fine but the problem is when the first >>> message arrive "dovecot-lda" return a "Segmentation fault", the >>> message is written to the user's Mailbox but the message remains, >>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>> the second attempt is delivered fine. >> gdb backtrace would be very helpful in figuring out the problem: >> http://dovecot.org/bugreport.html >> > > Obvious ... :-) > > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. > Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. > Loaded symbols for > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so > Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver > -d cecchi10 at qboxdns.it'. > Program terminated with signal 11, Segmentation fault. > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > 221 if (dict->dict == NULL) > > is enough? This is the full bt: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = 0x0, v_modifiable = 0x0} ids = 0x1928658 i = dest = ret = -883075307 #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x1944240) at acl-backend-vfile-acllist.c:297 auser = 0x1949a08 iter = 0x0 acllist_path = 0x1928658 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" ret = ns = 0x1943e50 output = 0x0 st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1348059559, tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x1928210 file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x1928610 "Sent" origin = 0x194d178 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh (backend=0x1944240) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x19444e0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1348059559, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 26515976 #6 0x00007f2fc9fc125e in acl_backend_get_default_rights (backend=0x1944240, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x1944240 acl_mask = 0x1 ns = 0x1943e50 update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = 0x7f2fcbf2ba88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ctx = 0x1946b20 pool = i = inboxcase = #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1348059559) at mailbox-list-iter.c:58 patterns = {0x7f2fc9db76dc "*", 0x0} #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:73 ctx = 0x7f2fcb5beef3 info = #12 quota_count (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff362e0038) at quota-dict.c:113 dt = bytes = 0 count = 0 #14 0x00007f2fc9db395a in dict_quota_update_callback ( ret=, context=0x1949a08) at quota-dict.c:178 value = 1 #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, line_r=) at dict-client.c:265 ctx = 0x1984190 #16 client_dict_read_one_line (dict=0x194c7f0, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f2fcb592565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x194c7f0 line = 0x0 ret = #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x1944950 #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x194c5c0 #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 quota = 0x1944720 i = 2 #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) at quota-storage.c:412 quser = 0x1949a30 quota_set = 0x1946f30 #22 0x00007f2fcb88188e in mail_user_unref (_user=) at mail-user.c:153 user = 0x19483c0 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x192e050, tried_default_save = true, saved_mail = true, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = user = 0x192d3c2 "cecchi10 at qboxdns.it" errstr = 0x0 path = 0x7fff362e0368 "\351\a" storage_service = 0x192f3a0 service_user = 0x192fd48 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x1943f30 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ret = c = error = MAIL_ERROR_NONE (gdb) Sorry -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From robert at schetterer.org Wed Sep 19 17:03:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:03:45 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125339.GG2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> Message-ID: <5059D0C1.5050202@schetterer.org> Am 19.09.2012 14:53, schrieb Jost Krieger: > On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > >> some allready existing way is i.e >> with milter and postfix tagged spam get in hold queue for i.e human >> admin investigate, or some quarantaine with amavis > > Human admin is not a good idea if you have 50000 users :-) > Quarantine's a possibility, but that's taking it out of "Mail" and adds > complexity. as ever ,this is different in different places specially Quarantine is wide used > >> for sure there are benefits getting dovecot involved in spam tagging >> but i would leave spam tagging to the mailserver in first way and >> deliver it to Junk folder by global sieve rule with giving more quota or >> ignoring Quota to Junk folder, but i see this isnt the same what you >> might wanna goal > > That's just what we want to do, *but* although it's in the Wiki I don't > think it a good idea to *raise* the limit for the junk folder if you > *deliver* to it. It's somewhat ok if you move mail there and someone > picks it up (we do that with a different folder). But on delivery it > means at some point you can receive spam but no real messages. Not nice. > > "ignore" may work if it does what I think (never even count bytes and > messages in this folder) but it is open to abuse ... there is no good or evil to this case, its a design question which may change in different places,, the abuse case might be acceptable, as the Junk folder i.e is imap special used autoconfigured and has auto empty feature, i give 50 mb plus quota on Junk folder and delete mail older then 3 month there > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 19 17:25:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:25:05 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <5059D5C1.500@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig > Hi Timo, i see a few rare errors with lucene and/or indexer since update /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out lucene-indexes) failed: Directory not empty Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) failed: Directory not empty lucene-indexes: IndexReader::open() failed (#1): File does not exist Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual method called Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called without an active exception Error: indexer failed to index mailbox INBOX should i care -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Wed Sep 19 17:25:30 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 16:25:30 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5059D0C1.5050202@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> Message-ID: <20120919142529.GI2010@ruhr-uni-bochum.de> On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > as ever ,this is different in different places > specially Quarantine is wide used Of course, it's often useful. > there is no good or evil to this case, its a design question which may > change in different places,, the abuse case might be acceptable, as the > Junk folder i.e is imap special used autoconfigured > and has auto empty feature, i give 50 mb plus quota on Junk folder > and delete mail older then 3 month there The abuse may be tolerable, but I just don't get how your +50M works. For me it would mean that someone with a large Junk folder just wouldn't get an more mail except something delivered straight to Junk. And if you don't deliver to Junk but the clients move stuff there, there's still a problem. I'm justing raising the limit for folders that clients use for deleting (like Trash or whatever it is called). Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From robert at schetterer.org Wed Sep 19 17:44:42 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:44:42 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919142529.GI2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> <20120919142529.GI2010@ruhr-uni-bochum.de> Message-ID: <5059DA5A.1070908@schetterer.org> Am 19.09.2012 16:25, schrieb Jost Krieger: > On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > >> as ever ,this is different in different places >> specially Quarantine is wide used > > Of course, it's often useful. > >> there is no good or evil to this case, its a design question which may >> change in different places,, the abuse case might be acceptable, as the >> Junk folder i.e is imap special used autoconfigured >> and has auto empty feature, i give 50 mb plus quota on Junk folder >> and delete mail older then 3 month there > > The abuse may be tolerable, but I just don't get how your +50M works. the situation isnt really comparable, the plus is more configured for copy by human to i.e Junk folder, or i.e a client filter does this auto it makes most sense with trash, so user should mostly able to copy to trash, even if they have got full quota, this should avoid some errors in clients http://wiki2.dovecot.org/Quota/Configuration ... quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = SPAM:ignore ... > For me it would mean that someone with a large Junk folder just wouldn't > get an more mail except something delivered straight to Junk. And if you > don't deliver to Junk but the clients move stuff there, with auto loosing mail after some time in my setup which is the anounced policy, Trash and Junk are "autodeleted" folders no mail there ,rests forever *g there's still a > problem. I'm justing raising the limit for folders that clients use for > deleting (like Trash or whatever it is called). as i said its a design question, your policy is ok too no flame ,having your idea as alternative is always fine to have > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From interfasys at gmail.com Wed Sep 19 17:55:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 15:55:33 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <5059DCE5.8030903@gmail.com> > Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. You are correct, it had not been compiled in a while. Here is a new backtrace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at user.c:96 _module_tmp = (void *) 0x10a84d18 asu = (struct antispam_user *) 0x10a84d18 tmp = Variable "tmp" is not available. Cheers, Olivier From tss at iki.fi Wed Sep 19 17:59:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 17:59:38 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5059DCE5.8030903@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> Message-ID: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > You are correct, it had not been compiled in a while. Here is a new > backtrace: .. > #6 0x00000000105e7a6a in buffer_write () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #7 0x00000000105ce9f9 in array_idx_set_i () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > user.c:96 > _module_tmp = (void *) 0x10a84d18 > asu = (struct antispam_user *) 0x10a84d18 > tmp = Variable "tmp" is not available. This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? From tss at iki.fi Wed Sep 19 18:02:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:02:07 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> Message-ID: <1348066927.13571.69.camel@hurina> On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: > On 19.9.2012, at 17.55, interfaSys s?rl wrote: > > >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > > > You are correct, it had not been compiled in a while. Here is a new > > backtrace: > .. > > #6 0x00000000105e7a6a in buffer_write () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #7 0x00000000105ce9f9 in array_idx_set_i () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > > user.c:96 > > _module_tmp = (void *) 0x10a84d18 > > asu = (struct antispam_user *) 0x10a84d18 > > tmp = Variable "tmp" is not available. > > This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? Attached a helpful antispam patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 444 bytes Desc: not available URL: From interfasys at gmail.com Wed Sep 19 18:12:01 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 16:12:01 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: <5059E0C1.9070906@gmail.com> You are once again correct. The update script had an error and thus the plugin was not recompiled for 2.1.10... Thank you! Olivier On 19/09/2012 16:02, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >>>> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. >>> >>> You are correct, it had not been compiled in a while. Here is a new >>> backtrace: >> .. >>> #6 0x00000000105e7a6a in buffer_write () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #7 0x00000000105ce9f9 in array_idx_set_i () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >>> user.c:96 >>> _module_tmp = (void *) 0x10a84d18 >>> asu = (struct antispam_user *) 0x10a84d18 >>> tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? > > Attached a helpful antispam patch. > From tss at iki.fi Wed Sep 19 18:16:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:16:58 +0300 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <5059D5C1.500@schetterer.org> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> Message-ID: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> On 19.9.2012, at 17.25, Robert Schetterer wrote: > Am 18.09.2012 20:52, schrieb Timo Sirainen: >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >> > > Hi Timo, i see a few rare errors with lucene and/or indexer > since update Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out > lucene-indexes) failed: Directory not empty > > Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) > failed: Directory not empty > > lucene-indexes: IndexReader::open() failed (#1): File does not exist The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual > method called > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called > without an active exception > > Error: indexer failed to index mailbox INBOX > > should i care > -- > Best Regards > MfG Robert Schetterer > From eugene at raptor.kiev.ua Wed Sep 19 19:13:13 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Wed, 19 Sep 2012 19:13:13 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On Wed, 19 Sep 2012 18:02:07 +0300, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >> >> Did you recompile antispam plugin? Is it the latest antispam? This >> line doesn't seem to match the current code. >> > >> > You are correct, it had not been compiled in a while. Here is a new >> > backtrace: >> .. >> > #6 0x00000000105e7a6a in buffer_write () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #7 0x00000000105ce9f9 in array_idx_set_i () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >> > user.c:96 >> > _module_tmp = (void *) 0x10a84d18 >> > asu = (struct antispam_user *) 0x10a84d18 >> > tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile >> antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is >> also really from v2.1.10? > > Attached a helpful antispam patch. Erm... Am I missing something? http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c line 26, or actually line 32 given http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jcblanco at fi.upm.es Wed Sep 19 19:22:29 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Wed, 19 Sep 2012 18:22:29 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058C22F.1070704@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> Message-ID: <5059F145.1030803@fi.upm.es> On 18/09/2012 20:49, Stephan Bosch wrote: > On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the >> new pigeonhole version, however I've got the follwing errors when >> compiling: >> >> It seems that the error is related to the ld.gold fix, If I revert the >> c52a0c561311 patch the error is not produced > > Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, > but I forgot the Dovecot v2.1 tree. Fixed now: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > This only happens when you compile against installed Dovecot headers, > people using --with-dovecot= (like me) would not have seen this. > > I'll push out a new release once 2.1.10 comes out. I'm being told that > is imminent. Sorry, but using the new 0.3.3 version I'm getting the same error and I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in configure, not the installed headers. If needed I can send you the configure invocation command. Regards Juan C. Blanco > > Regards, > > Stephan. > > From tss at iki.fi Wed Sep 19 19:24:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 19:24:22 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On 19.9.2012, at 19.13, Eugene Paskevich wrote: >>> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? >> >> Attached a helpful antispam patch. > > Erm... Am I missing something? > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c > line 26, or actually line 32 given > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 Oh, didn't notice it there below :) Wonder why it then didn't complain about version mismatch. From alec at alec.pl Wed Sep 19 20:10:16 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 19 Sep 2012 19:10:16 +0200 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> Message-ID: <5059FC78.7090100@alec.pl> On 09/19/2012 02:42 PM, Timo Sirainen wrote: > mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. I'm confused. I have no problems with creating and using a folder with utf8 name containing '&' character. So, what problem do you describe? doveadm is working with utf8 right? So, why it can't delete such folder. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From florob at babelmonkeys.de Wed Sep 19 20:43:09 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 19:43:09 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> Message-ID: <505A042D.5090705@babelmonkeys.de> Am 11.09.2012 20:21, schrieb Timo Sirainen: > On 11.9.2012, at 21.07, Florian Zeitz wrote: > >>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>> >> Well, either that, or we could use a union of all known context structs >> there. > > That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. > >> Or we could use i_malloc() and i_free() under the assumption hmac_init() >> + hmac_final() calls are always matched. > > Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. > Here is an updated version of the patch. It uses the variant defining the largest allowed content size. I choose 1024 bit (128 Byte), since that is what most of the SHA-3 candidates use, so it's hopefully somewhat future proof. It's also rebased to the current head and switches the hmac uses added since the last version over to the changed API. It also fixes an "interesting" compiler warning I got with clang, which complained about the fact that struct hmac_context* was (re)declared in the prototype of hmac_md5_get_cram_context() and therefore separate from the one declared in hmac.h. Regards, Florian -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 0aa0a656176c9ca1c3f187999cd4725bf52665e5 # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE (1024 / 8) + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From interfasys at gmail.com Wed Sep 19 20:52:20 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 18:52:20 +0100 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <505A0654.3010501@gmail.com> I see the same crashes on FreeBSD. Turning off ACL makes the crashes disappear. I'm getting a different backtrace though and it doesn't seem to be related to ACL: Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libssl.so.8...done. Loaded symbols for /usr/local/lib/libssl.so.8 Reading symbols from /usr/local/lib/libcrypto.so.8...done. Loaded symbols for /usr/local/lib/libcrypto.so.8 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /lib/libthr.so.3...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/lib/libz.so...done. Loaded symbols for /usr/lib/libz.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 143 return (_c < 0 || _c >= __mb_sb_limit) ? _c : [New Thread 10e041c0 (LWP 103332/initial thread)] (gdb) bt full #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 table = (struct hash_table *) 0x10e29700 #1 0x0000000010478cf1 in settings_parser_deinit (_ctx=0x10e29778) at array.h:197 ctx = (struct setting_parser_context *) 0x0 #2 0x00000000104ac4ea in master_service_settings_cache_deinit (_cache=0x1045da08) at master-service-settings-cache.c:81 cache = (struct master_service_settings_cache *) 0x10e2e040 entry = (struct settings_entry *) 0x10e29740 next = (struct settings_entry *) 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x000000001044d1df in login_settings_deinit () at login-settings.c:129 No locals. #4 0x000000001044dd7e in main_deinit () at main.c:62 No locals. #5 0x000000001044df5c in login_binary_run (binary=0x4051c0, argc=2, argv=0x7fffffffebd8) at main.c:62 service_flags = 72 set_pool = 0x10e2d020 allow_core_dumps = true login_socket = 0x10457491 "login" c = -1 #6 0x00000000004036ee in main (argc=2, argv=0x7fffffffebd8) at client.c:40 No locals. Cheers, Olivier From stephan at rename-it.nl Wed Sep 19 21:07:12 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 19 Sep 2012 20:07:12 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5059F145.1030803@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> Message-ID: <505A09D0.6040607@rename-it.nl> On 9/19/2012 6:22 PM, Juan C. Blanco wrote: > On 18/09/2012 20:49, Stephan Bosch wrote: >> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>> It seems that the error is related to the ld.gold fix, If I revert the >>> c52a0c561311 patch the error is not produced >> >> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >> but I forgot the Dovecot v2.1 tree. Fixed now: >> >> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c D'oh, this is the v2.2 change. The URL I meant is this one: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >> >> This only happens when you compile against installed Dovecot headers, >> people using --with-dovecot= (like me) would not have seen this. >> >> I'll push out a new release once 2.1.10 comes out. I'm being told that >> is imminent. > > Sorry, but using the new 0.3.3 version I'm getting the same error and > I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in > configure, not the installed headers. If needed I can send you the > configure invocation command. Yes, that could be helpful. Regards, Stephan. From florob at babelmonkeys.de Wed Sep 19 21:42:50 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 20:42:50 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <505A042D.5090705@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> <505A042D.5090705@babelmonkeys.de> Message-ID: <505A122A.3070307@babelmonkeys.de> Am 19.09.2012 19:43, schrieb Florian Zeitz: > Am 11.09.2012 20:21, schrieb Timo Sirainen: >> On 11.9.2012, at 21.07, Florian Zeitz wrote: >> >>>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>>> >>> Well, either that, or we could use a union of all known context structs >>> there. >> >> That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. >> >>> Or we could use i_malloc() and i_free() under the assumption hmac_init() >>> + hmac_final() calls are always matched. >> >> Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. >> > Here is an updated version of the patch. > It uses the variant defining the largest allowed content size. > I choose 1024 bit (128 Byte), since that is what most of the SHA-3 > candidates use, so it's hopefully somewhat future proof. Forget I said that. Unfortunately the MD5 context is already bigger (the SHA-1 one isn't). So 256 Byte it is I guess. Sorry for the lack of thorough testing. I hope I didn't miss another hash function. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 5e6802858a5bb3a9b278f1cbdabe5c0ffb15248d # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE 256 + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From c at roessner-network-solutions.com Wed Sep 19 22:48:17 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 21:48:17 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: Hi, > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 unfortunately z-Push is much too buggy here. So I dropped it in favor of standard IMAP. Best wishes -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From ben at indietorrent.org Thu Sep 20 00:41:38 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 17:41:38 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504F4D5D.8040207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> Message-ID: <505A3C12.7050207@indietorrent.org> On 9/11/2012 10:40 AM, Ben Johnson wrote: > You nailed it, Ben. Thanks! > > I downloaded/installed the dovecot-dev package from the same maintainer > and was able to build Antispam without issue. > > The hang-up now seems to be with my configuration. > > I'm seeing the following error in my email client whenever I attempt to > move an email message from, say, Inbox to Junk folder: > > [SERVERBUG] BUG Unknown internal error. > > Nothing at all appears in syslog when this happens. > > I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam > , to the letter, I believe. > > The relevant portions of my configuration are as follows: > > --------------------- > protocol imap { > mail_plugins = quota imap_quota antispam > } > > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > > antispam_backend = MAILTRAIN > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > antispam_spam_pattern_ignorecase = spam;junk > } > --------------------- > > The permissions and execute bit are set in accordance with other > executables in the same directory: > > # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" > -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh > > I would provide more log content, but the debug options seem to be gone > in the forked, >= 2.0 version of this plug-in. > > Any ideas are very much appreciated. > > Thanks, > > -Ben Given that nobody shared any insight here, I decided to stand-up a completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. While the error is different this time around, I am still unable to get any variant of the dovecot-antispam plug-in working. When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and I use the above configuration, my syslog fills-up with the following: dovecot: imap-login: Login: user=, method=PLAIN, rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS dovecot: master: Error: service(imap): child 2879 returned error 3 Surprisingly, I can find only two other mentions of "error 3" anywhere on the Internet. Those mentions were from another user to this mailing list: http://www.dovecot.org/list/dovecot/2012-August/067787.html http://www.dovecot.org/list/dovecot/2012-August/067814.html The lack of "error 3" ubiquity leads me to believe that I must be doing something stupid. There's no question that this behavior is related to the antispam-plugin, because when I commented-out the relevant lines in Dovecot's configuration file, the error does not occur. I have followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the result. The only instance in which I diverged from the Wiki article is that I used the Ubuntu-provided dovecot-antispam package (instead of compiling the plug-in from source). I am trying to use the "mailtrain" back-end, per the above-cited Wiki article, to call SpamAssassin's "sa-learn" utility for real-time learning. I am not using the SpamAssassin daemon (spamd). I'm about to throw in the towel on this plug-in, so any help would be greatly appreciated. Thank you, -Ben From tom at whyscream.net Thu Sep 20 01:19:07 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 20 Sep 2012 00:19:07 +0200 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505A44DB.6020200@whyscream.net> On 19/09/12 23:41, Ben Johnson wrote: > > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben There are 2 versions of the dovecot-antispam package. The original version was created by Johannes Berg, then died almost a silent death, and was forked by Eugene Paskevitch. Later Johannes revived his original version somewhat. The ubuntu package is based on the 'revived Johannes fork' [1], while the documentation on the dovecot wiki describes the 'Eugene fork'. [2] [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright [2] http://wiki2.dovecot.org/Plugins/Antispam Thus you'll need to choose which plugin you want, then follow the complete procedure for that plugin only. They both can do what you want, but building and configuration are slightly different. It would be nice if Johannes and Eugene would either agree on a single version of the plugin, or rename one of the forks so they don't get mixed up that easily. I'm not sure if that has been discussed (on- or off-list) in the past. FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to upgrade to 2.1 yet, so no experience there. -- Tom From ben at indietorrent.org Thu Sep 20 02:17:43 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 19:17:43 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A44DB.6020200@whyscream.net> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> Message-ID: <505A5297.8050808@indietorrent.org> On 9/19/2012 6:19 PM, Tom Hendrikx wrote: > On 19/09/12 23:41, Ben Johnson wrote: >> >> >> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>> You nailed it, Ben. Thanks! >>> >>> I downloaded/installed the dovecot-dev package from the same maintainer >>> and was able to build Antispam without issue. >>> >>> The hang-up now seems to be with my configuration. >>> >>> I'm seeing the following error in my email client whenever I attempt to >>> move an email message from, say, Inbox to Junk folder: >>> >>> [SERVERBUG] BUG Unknown internal error. >>> >>> Nothing at all appears in syslog when this happens. >>> >>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>> , to the letter, I believe. >>> >>> The relevant portions of my configuration are as follows: >>> >>> --------------------- >>> protocol imap { >>> mail_plugins = quota imap_quota antispam >>> } >>> >>> plugin { >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> sieve = /var/vmail/%d/%n/.sieve >>> >>> antispam_backend = MAILTRAIN >>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>> antispam_mail_spam = --spam >>> antispam_mail_notspam = --ham >>> antispam_spam_pattern_ignorecase = spam;junk >>> } >>> --------------------- >>> >>> The permissions and execute bit are set in accordance with other >>> executables in the same directory: >>> >>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>> >>> I would provide more log content, but the debug options seem to be gone >>> in the forked, >= 2.0 version of this plug-in. >>> >>> Any ideas are very much appreciated. >>> >>> Thanks, >>> >>> -Ben >> >> Given that nobody shared any insight here, I decided to stand-up a >> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >> >> While the error is different this time around, I am still unable to get >> any variant of the dovecot-antispam plug-in working. >> >> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >> I use the above configuration, my syslog fills-up with the following: >> >> dovecot: imap-login: Login: user=, method=PLAIN, >> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >> dovecot: master: Error: service(imap): child 2879 returned error 3 >> >> Surprisingly, I can find only two other mentions of "error 3" anywhere >> on the Internet. Those mentions were from another user to this mailing list: >> >> http://www.dovecot.org/list/dovecot/2012-August/067787.html >> http://www.dovecot.org/list/dovecot/2012-August/067814.html >> >> The lack of "error 3" ubiquity leads me to believe that I must be doing >> something stupid. >> >> There's no question that this behavior is related to the >> antispam-plugin, because when I commented-out the relevant lines in >> Dovecot's configuration file, the error does not occur. >> >> I have followed the instructions at >> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >> result. >> >> The only instance in which I diverged from the Wiki article is that I >> used the Ubuntu-provided dovecot-antispam package (instead of compiling >> the plug-in from source). >> >> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >> >> I am not using the SpamAssassin daemon (spamd). >> >> I'm about to throw in the towel on this plug-in, so any help would be >> greatly appreciated. >> >> Thank you, >> >> -Ben > > There are 2 versions of the dovecot-antispam package. The original > version was created by Johannes Berg, then died almost a silent death, > and was forked by Eugene Paskevitch. Later Johannes revived his original > version somewhat. > > The ubuntu package is based on the 'revived Johannes fork' [1], while > the documentation on the dovecot wiki describes the 'Eugene fork'. [2] > > [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright > [2] http://wiki2.dovecot.org/Plugins/Antispam > > Thus you'll need to choose which plugin you want, then follow the > complete procedure for that plugin only. They both can do what you want, > but building and configuration are slightly different. > > It would be nice if Johannes and Eugene would either agree on a single > version of the plugin, or rename one of the forks so they don't get > mixed up that easily. I'm not sure if that has been discussed (on- or > off-list) in the past. > > FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the > 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to > upgrade to 2.1 yet, so no experience there. > > -- > Tom > Thank you for your timely response, Tom. Much appreciated. Your remarks clear-up a lot of the confusion for me. I didn't realize that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did I realize that this is the version that is packaged with Ubuntu 12.04. Given that I'm more-or-less forced to work with Ubuntu LTS, I would prefer to use the OS's package management system wherever possible. I suppose, by extension, that I would prefer to use the "revived Johannes fork". I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but am unable to get either to work the way I want (mailtrain back-end with real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki article). Shouldn't the basic tenets of each version of the plug-in apply? Where should I start, in the way of seeking support? Basically, I intend to use the Johannes version, but I want to configure it in the manner described in the Eugene version Wiki article. I've already described (rather exhaustively) what happens when I attempt to use the Johannes version and adapt the Eugene version instructions. I'm curious to know what method you've used over the years with such glowing success. Which back-end did you use with Dovecot 1/Johannes? How about with Dovecot 2/Eugene? Thanks again, -Ben From selcuk.yazar at gmail.com Thu Sep 20 09:21:21 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:21:21 +0300 Subject: [Dovecot] Prevent Download messages from server Message-ID: Hi, can we prevent download messages from server user by user ? sme common used mail's message must be remain at the server, but sometimes we download them ? thanks in advance -- Sel?uk YAZAR From robert at schetterer.org Thu Sep 20 09:28:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:28:37 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AB795.3000106@schetterer.org> Am 20.09.2012 08:21, schrieb Selcuk Yazar: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > please clear , what you mean with 1 : with download messages 2 : user by user 3 : sme 4 : mail's message must be remain at the server 5 : but sometimes we download them show your dovecot conf and problem logs -- Best Regards MfG Robert Schetterer From selcuk.yazar at gmail.com Thu Sep 20 09:31:19 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:31:19 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AB795.3000106@schetterer.org> References: <505AB795.3000106@schetterer.org> Message-ID: Hi, we have no problem, just i want to learn how can i do that. i think it's clear . sel?uk On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > Am 20.09.2012 08:21, schrieb Selcuk Yazar: > > Hi, > > > > can we prevent download messages from server user by user ? sme common > used > > mail's message must be remain at the server, but sometimes we download > them > > ? > > > > thanks in advance > > > > please clear , what you mean with > > 1 : with download messages > 2 : user by user > 3 : sme > 4 : mail's message must be remain at the server > 5 : but sometimes we download them > > show your dovecot conf and problem logs > > -- > Best Regards > MfG Robert Schetterer > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From robert at schetterer.org Thu Sep 20 09:38:48 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:38:48 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AB795.3000106@schetterer.org> Message-ID: <505AB9F8.6060603@schetterer.org> Am 20.09.2012 08:31, schrieb Selcuk Yazar: > Hi, > > we have no problem, just i want to learn how can i do that. i think it's > clear . > > > sel?uk ok ,so the right answer is, unless you give more details stop dovecot , nobody will able to connect to it > > On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > >> Am 20.09.2012 08:21, schrieb Selcuk Yazar: >>> Hi, >>> >>> can we prevent download messages from server user by user ? sme common >> used >>> mail's message must be remain at the server, but sometimes we download >> them >>> ? >>> >>> thanks in advance >>> >> >> please clear , what you mean with >> >> 1 : with download messages >> 2 : user by user >> 3 : sme >> 4 : mail's message must be remain at the server >> 5 : but sometimes we download them >> >> show your dovecot conf and problem logs >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > -- Best Regards MfG Robert Schetterer From alessio at skye.it Thu Sep 20 09:45:03 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:45:03 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ABB6F.2010606@skye.it> Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > If your users can log and read messages can obviously download it. If you use IMAP you can set ACL and enable read only for the folders: http://wiki2.dovecot.org/ACL -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Thu Sep 20 09:51:14 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:51:14 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <505A0654.3010501@gmail.com> References: <505A0654.3010501@gmail.com> Message-ID: <505ABCE2.30802@skye.it> Il 19/09/2012 19:52, interfaSys s?rl ha scritto: > I see the same crashes on FreeBSD. Turning off ACL makes the crashes > disappear. > > I'm getting a different backtrace though and it doesn't seem to be > related to ACL: > > Core was generated by `imap-login'. > Thanks for your report. My backtrace was genereted from "dovecot-lda", but in both cases disable ACL "solve" the problem. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From selcuk.yazar at gmail.com Thu Sep 20 09:53:28 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:53:28 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ABB6F.2010606@skye.it> References: <505ABB6F.2010606@skye.it> Message-ID: Thank you, i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think it's diffucult to evaluate in POP3. also we are using openLDAP backend, if i change folder permissions , does it work ? sel?uk On Thu, Sep 20, 2012 at 9:45 AM, Alessio Cecchi wrote: > Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > >> Hi, >> >> can we prevent download messages from server user by user ? sme common >> used >> mail's message must be remain at the server, but sometimes we download >> them >> ? >> >> thanks in advance >> >> > If your users can log and read messages can obviously download it. > > If you use IMAP you can set ACL and enable read only for the folders: > > http://wiki2.dovecot.org/ACL > > -- > Alessio Cecchi is: > @ ILS -> http://www.linux.it/~alessice/ > on LinkedIn -> http://www.linkedin.com/in/**alessice > Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ > @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From odhiambo at gmail.com Thu Sep 20 10:13:08 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 20 Sep 2012 10:13:08 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: On Thu, Sep 20, 2012 at 9:53 AM, Selcuk Yazar wrote: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Changing folder permissions to what? I suppose you understand what you mean by changing folder permissions! If you make them read-only (no write) for the users group, as that is what you could be thinking, you'll probably also affect the ability of the MTA to write to them (deliver mail). Please explain your situation holistically so that we can understand and give suggestions... -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. From alessio at skye.it Thu Sep 20 10:19:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 09:19:31 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: <505AC383.2010107@skye.it> Il 20/09/2012 08:53, Selcuk Yazar ha scritto: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Change folder permissions is a bad idea, for example if you use Maildir/ as mailbox format you can't remove +w, messages should be moved from new/ to cur/, add flags to name and so on. I do not think it's possible to do what you ask. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From apm at one.com Thu Sep 20 10:24:34 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 20 Sep 2012 09:24:34 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AC4B2.2040900@one.com> > we have no problem, just i want to learn how can i do that. i think > it's clear . Well... I'm pretty sure most others don't. But anyway. As in ALL Internet protocols (IMAP being no exception), letting the client read data on the server requires it to download the data. Preventing download will prevent reading the mail. Period. So if you're fine with that and just want to learn how to do it, then just disable the account in the user database. /Peter From selcuk.yazar at gmail.com Thu Sep 20 10:28:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:28:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: Alessio and Peter thank you again. maybe it can be future request. :) selcuk. On Thu, Sep 20, 2012 at 10:24 AM, Peter Mogensen wrote: > > we have no problem, just i want to learn how can i do that. i think > > it's clear . > > Well... I'm pretty sure most others don't. > > But anyway. As in ALL Internet protocols (IMAP being no exception), > letting the client read data on the server requires it to download the data. > Preventing download will prevent reading the mail. Period. > > So if you're fine with that and just want to learn how to do it, then just > disable the account in the user database. > > /Peter > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From jc at info-systems.de Thu Sep 20 10:51:41 2012 From: jc at info-systems.de (Jakob Curdes) Date: Thu, 20 Sep 2012 09:51:41 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AC4B2.2040900@one.com> Message-ID: <505ACB0D.7020600@info-systems.de> Am 20.09.2012 09:28, schrieb Selcuk Yazar: > Alessio and Peter thank you again. maybe it can be future request. :) We are trying to tell you that it is impossible to write a mail _client_ that does not download (i.e. transfer) the mail that it shows to the user. This is why it makes no sense to prevent the download on the server side, and so there will never be such a function. Preventing the download (the transfer) of the mail fro mthe server to the client just means that the user *cannot read it*. Another question is whether there are mail _clients_ which do not offer a possibility to *save* the read mail (which has already been "downloaded" for reading it). But that question is not really related to the dovecot mail server. If you want to get users to read mails, but not download them, I would suggest you deny the direct access to the mail server and use a browser-based application that shows the entire mail to the user, maybe as a graphical representation. Or you use some sort of terminal software that connects to a host where the mail _clients_ itself are installed, just letting the user view the data but nor copy it, e.g. the VNC protocol. Hope this helps, Jakob Curdes From selcuk.yazar at gmail.com Thu Sep 20 10:54:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:54:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ACB0D.7020600@info-systems.de> References: <505AC4B2.2040900@one.com> <505ACB0D.7020600@info-systems.de> Message-ID: why are you trying ??? this is brain storming On Thu, Sep 20, 2012 at 10:51 AM, Jakob Curdes wrote: > Am 20.09.2012 09:28, schrieb Selcuk Yazar: > > Alessio and Peter thank you again. maybe it can be future request. :) > > We are trying to tell you that it is impossible to write a mail *client*that does not download (i.e. transfer) the mail that it shows to the user. > This is why it makes no sense to prevent the download on the server side, > and so there will never be such a function. > Preventing the download (the transfer) of the mail fro mthe server to the > client just means that the user *cannot read it*. > > Another question is whether there are mail *clients* which do not offer a > possibility to *save* the read mail (which has already been "downloaded" > for reading it). But that question is not really related to the dovecot > mail server. > > If you want to get users to read mails, but not download them, I would > suggest you deny the direct access to the mail server and use a > browser-based application that shows the entire mail to the user, maybe as > a graphical representation. Or you use some sort of terminal software > that connects to a host where the mail *clients* itself are installed, > just letting the user view the data but nor copy it, e.g. the VNC protocol. > > Hope this helps, > Jakob Curdes > > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From Bill at knoxvillechristian.org Thu Sep 20 11:32:54 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 20 Sep 2012 04:32:54 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505AD4B6.1020109@knoxvillechristian.org> On 9/19/2012 5:41 PM, Ben Johnson wrote: > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben I've never used this plugin but let offer up my home brewed script that I call from crontab: # Process and delete .Spam and .Ham Maildirs 11 */4 * * * /usr/local/sbin/maildir.sa-learn and /usr/local/sbin/maildir.sa-learn: #!/bin/bash MAILDIR=Maildir LS=/bin/ls RM=/bin/rm SU=/bin/su FIND=/bin/find SALEARN=/usr/bin/sa-learn for i in `$LS /home`; do prefs="/home/$i/.spamassassin/user_prefs" # [ ! ."$i" == ".bill" ] && continue echo $i spamdir="/home/$i/$MAILDIR/.SystemFolders.Spam" [ ! -d "$spamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/new" $i $FIND "$spamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/cur" $i $FIND "$spamdir/cur" -type f -delete hamdir="/home/$i/$MAILDIR/.SystemFolders.Ham" [ ! -d "$hamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/new" $i $FIND "$hamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/cur" $i $FIND "$hamdir/cur" -type f -delete done With my setup each user has their own spamassassin user_prefs and bayes tokens. Bill From robert at schetterer.org Thu Sep 20 11:42:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 10:42:05 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: <505AD6DD.3020700@schetterer.org> Am 20.09.2012 09:24, schrieb Peter Mogensen: > Preventing download will prevent reading the mail famous last words *g -- Best Regards MfG Robert Schetterer From david at blue-labs.org Thu Sep 20 12:16:06 2012 From: david at blue-labs.org (David Ford) Date: Thu, 20 Sep 2012 05:16:06 -0400 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ADED6.7030204@blue-labs.org> do you mean to leave a copy of the email on the server so it can be read in multiple email clients? IMAP can do this and i think modern POP3 can. look for an account config option in your mail client to "leave mail on server". i think there is a setting in dovecot to prevent expunging of email but it has been years since i was researching this. -david On 09/20/2012 02:21 AM, Selcuk Yazar wrote: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > From robert at schetterer.org Thu Sep 20 17:51:04 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 16:51:04 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> Message-ID: <505B2D58.3010301@schetterer.org> Am 19.09.2012 17:16, schrieb Timo Sirainen: > On 19.9.2012, at 17.25, Robert Schetterer wrote: > >> Am 18.09.2012 20:52, schrieb Timo Sirainen: >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >>> >> >> Hi Timo, i see a few rare errors with lucene and/or indexer >> since update > > Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > >> /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out >> lucene-indexes) failed: Directory not empty >> >> Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) >> failed: Directory not empty >> >> lucene-indexes: IndexReader::open() failed (#1): File does not exist > > The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual >> method called >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called >> without an active exception >> >> Error: indexer failed to index mailbox INBOX >> >> should i care >> -- >> Best Regards >> MfG Robert Schetterer >> > update: the above errors didnt happen again today for one time this error comes up instead Error: Log synchronization error at seq=8,offset=28596 for /usr/local/virtual/...../...../dovecot.index: Extension header update without intro prefix -- Best Regards MfG Robert Schetterer From surreal at bsdmail.com Thu Sep 20 18:24:16 2012 From: surreal at bsdmail.com (Surreal) Date: Thu, 20 Sep 2012 23:24:16 +0800 Subject: [Dovecot] userdb missing driver Message-ID: <505B3520.10309@bsdmail.com> Good Day, I'm using dovecot 2.1.8 and this is my dovecot -n output: # 2.1.8: dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = domain.xxx auth_mechanisms = plain login auth_realms = domainone.xxx domaintwo.xxx disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver ---------------------------- This is my dovecot-sql.conf.ext connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix driver = mysql default_pass_scheme = MD5-CRYPT password_query = SELECT password,CONCAT('/mail/', maildir) AS userdb_home,\ '5000' AS userdb_uid, '5000' AS userdb_gid, allow_nets,\ concat('*:bytes=', quota) AS userdb_quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 user_query = SELECT CONCAT('/mail/', maildir) AS home, 'maildir:~/' as mail, '5000' AS uid, '5000' AS gid,\ concat('*:bytes=', quota) AS quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 #iterate_query = SELECT username AS user FROM mailbox ------------------------------ I'm sure i granted all privileges to daemonuser to postfix db. This is my first on dovecot so my knowledge is very minimal, all help possible is very much appreciated. Thanks From h.reindl at thelounge.net Thu Sep 20 18:43:55 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 20 Sep 2012 17:43:55 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3520.10309@bsdmail.com> References: <505B3520.10309@bsdmail.com> Message-ID: <505B39BB.3060308@thelounge.net> Am 20.09.2012 17:24, schrieb Surreal: > Good Day, > > I'm using dovecot 2.1.8 and this is my dovecot -n output: > > # 2.1.8: dovecot.conf > # OS: FreeBSD 9.0-STABLE i386 > > doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver > > connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix > driver = mysql no idea about BSD but on linux systems these are typically seperated packages dovecot-2.1.10-4.fc17.20120919.rh.x86_64 dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ben at indietorrent.org Thu Sep 20 18:53:12 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 11:53:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A5297.8050808@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> Message-ID: <505B3BE8.5050202@indietorrent.org> On 9/19/2012 7:17 PM, Ben Johnson wrote: > > > On 9/19/2012 6:19 PM, Tom Hendrikx wrote: >> On 19/09/12 23:41, Ben Johnson wrote: >>> >>> >>> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>>> You nailed it, Ben. Thanks! >>>> >>>> I downloaded/installed the dovecot-dev package from the same maintainer >>>> and was able to build Antispam without issue. >>>> >>>> The hang-up now seems to be with my configuration. >>>> >>>> I'm seeing the following error in my email client whenever I attempt to >>>> move an email message from, say, Inbox to Junk folder: >>>> >>>> [SERVERBUG] BUG Unknown internal error. >>>> >>>> Nothing at all appears in syslog when this happens. >>>> >>>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>>> , to the letter, I believe. >>>> >>>> The relevant portions of my configuration are as follows: >>>> >>>> --------------------- >>>> protocol imap { >>>> mail_plugins = quota imap_quota antispam >>>> } >>>> >>>> plugin { >>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>> sieve = /var/vmail/%d/%n/.sieve >>>> >>>> antispam_backend = MAILTRAIN >>>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>>> antispam_mail_spam = --spam >>>> antispam_mail_notspam = --ham >>>> antispam_spam_pattern_ignorecase = spam;junk >>>> } >>>> --------------------- >>>> >>>> The permissions and execute bit are set in accordance with other >>>> executables in the same directory: >>>> >>>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>>> >>>> I would provide more log content, but the debug options seem to be gone >>>> in the forked, >= 2.0 version of this plug-in. >>>> >>>> Any ideas are very much appreciated. >>>> >>>> Thanks, >>>> >>>> -Ben >>> >>> Given that nobody shared any insight here, I decided to stand-up a >>> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >>> >>> While the error is different this time around, I am still unable to get >>> any variant of the dovecot-antispam plug-in working. >>> >>> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >>> I use the above configuration, my syslog fills-up with the following: >>> >>> dovecot: imap-login: Login: user=, method=PLAIN, >>> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >>> dovecot: master: Error: service(imap): child 2879 returned error 3 >>> >>> Surprisingly, I can find only two other mentions of "error 3" anywhere >>> on the Internet. Those mentions were from another user to this mailing list: >>> >>> http://www.dovecot.org/list/dovecot/2012-August/067787.html >>> http://www.dovecot.org/list/dovecot/2012-August/067814.html >>> >>> The lack of "error 3" ubiquity leads me to believe that I must be doing >>> something stupid. >>> >>> There's no question that this behavior is related to the >>> antispam-plugin, because when I commented-out the relevant lines in >>> Dovecot's configuration file, the error does not occur. >>> >>> I have followed the instructions at >>> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >>> result. >>> >>> The only instance in which I diverged from the Wiki article is that I >>> used the Ubuntu-provided dovecot-antispam package (instead of compiling >>> the plug-in from source). >>> >>> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >>> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >>> >>> I am not using the SpamAssassin daemon (spamd). >>> >>> I'm about to throw in the towel on this plug-in, so any help would be >>> greatly appreciated. >>> >>> Thank you, >>> >>> -Ben >> >> There are 2 versions of the dovecot-antispam package. The original >> version was created by Johannes Berg, then died almost a silent death, >> and was forked by Eugene Paskevitch. Later Johannes revived his original >> version somewhat. >> >> The ubuntu package is based on the 'revived Johannes fork' [1], while >> the documentation on the dovecot wiki describes the 'Eugene fork'. [2] >> >> [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright >> [2] http://wiki2.dovecot.org/Plugins/Antispam >> >> Thus you'll need to choose which plugin you want, then follow the >> complete procedure for that plugin only. They both can do what you want, >> but building and configuration are slightly different. >> >> It would be nice if Johannes and Eugene would either agree on a single >> version of the plugin, or rename one of the forks so they don't get >> mixed up that easily. I'm not sure if that has been discussed (on- or >> off-list) in the past. >> >> FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the >> 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to >> upgrade to 2.1 yet, so no experience there. >> >> -- >> Tom >> > > Thank you for your timely response, Tom. Much appreciated. > > Your remarks clear-up a lot of the confusion for me. I didn't realize > that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did > I realize that this is the version that is packaged with Ubuntu 12.04. > > Given that I'm more-or-less forced to work with Ubuntu LTS, I would > prefer to use the OS's package management system wherever possible. I > suppose, by extension, that I would prefer to use the "revived Johannes > fork". > > I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but > am unable to get either to work the way I want (mailtrain back-end with > real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki > article). Shouldn't the basic tenets of each version of the plug-in apply? > > Where should I start, in the way of seeking support? Basically, I intend > to use the Johannes version, but I want to configure it in the manner > described in the Eugene version Wiki article. > > I've already described (rather exhaustively) what happens when I attempt > to use the Johannes version and adapt the Eugene version instructions. > > I'm curious to know what method you've used over the years with such > glowing success. Which back-end did you use with Dovecot 1/Johannes? How > about with Dovecot 2/Eugene? > > Thanks again, > > -Ben > Thanks again, Tom. After you set me straight on the various versions of the plug-in, I was able to synthesize the relevant documentation and make it all work on Ubuntu 12.04 LTS with Dovecot 2. The key was to understand the following: 1.) The Ubuntu repositories use the Johannes versions of the plug-in (for both Dovecot 1 and 2). Ubuntu 10.04 LTS (Dovecot 1) http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html Ubuntu 12.04 LTS (Dovecot 2) http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to the Eugene version, and that version's configuration directives differ from the Johannes version 2 directives. So, anyone attempting to follow the Wiki article on Ubuntu will need to adapt the directives accordingly (see documentation links above for equivalent directive names). 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki instructions lacks a shebang; until a proper shebang is added to the shell script, script execution will fail with "status 1". I added the standard shebang: #!/bin/sh Here are the configuration directives that work for me: plugin { antispam_debug_target = syslog antispam_backend = pipe antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_pipe_tmpdir = /tmp antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham } My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. I'm using the following directives, but when I move mail from Inbox to Junk, for example, I see no evidence that antispam is taking action. plugin { antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_mail_tmpdir = /tmp antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham } Dovecot writes nothing at all to syslog. No errors, no reference to antispam. And the message is moved successfully (no failure/error notice in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", attempts to create a log file in /tmp, which works with Antispam 2. But no such luck here; nothing is ever written to /tmp. Version 2 of the plug-in includes a verbose debugging mode, but it seems that no such feature is present in version 1. I noticed also that version 1 seems not to require a back-end to be specified explicitly (as in version 2). Is the back-end implied via the configuration directives that are present? Respectfully, -Ben From f.bonnet at esiee.fr Thu Sep 20 18:57:17 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 20 Sep 2012 17:57:17 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B39BB.3060308@thelounge.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> Message-ID: <505B3CDD.6@esiee.fr> On 09/20/2012 05:43 PM, Reindl Harald wrote: > Am 20.09.2012 17:24, schrieb Surreal: >> Good Day, >> >> I'm using dovecot 2.1.8 and this is my dovecot -n output: >> >> # 2.1.8: dovecot.conf >> # OS: FreeBSD 9.0-STABLE i386 >> >> doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver >> >> connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix >> driver = mysql > no idea about BSD but on linux systems these are typically > seperated packages > > dovecot-2.1.10-4.fc17.20120919.rh.x86_64 > dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 > > > if you install from FreeBSD ports you need to enable the MYSQL backend during configuration ( make config ) From tss at iki.fi Thu Sep 20 19:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 20 Sep 2012 18:01:05 +0200 Subject: [Dovecot] doveadm with multiple commands Message-ID: This has been in my TODO for a while and today I implemented it (but didn't dare to test it yet :) So the problem has been that you want to run for example: doveadm expunge -A mailbox Trash savedbefore 7d doveadm purge -A This is annoying because it scans through the users' mailboxes twice, wasting disk IO and CPU. So how to make it do the scan only once? The syntax I thought of today would be: doveadm multi [-A | -u wildcards] [ [...]] So the first two commands could be for example: doveadm multi -A : expunge mailbox Trash savedbefore 7d : purge The ':' character is the separator. Any other string could be used as well, for example "?SEP?" or whatever. Thoughts? Any better name for the command than "multi"? From horsley1953 at gmail.com Thu Sep 20 15:11:26 2012 From: horsley1953 at gmail.com (Tom Horsley) Date: Thu, 20 Sep 2012 08:11:26 -0400 Subject: [Dovecot] Sieve as mail client? Message-ID: <20120920081126.0627cd71@tomh> A slightly off topic question, but maybe someone here has seen something like this: I want a program that operates entirely on the client side of an IMAP server, and implements the sieve filtering language by doing the necessary client side operations to move mail on the IMAP server, expunge mail, etc. It would also act as a proxy between my real mail client and the IMAP server so I'd never see any mail that hasn't been filtered first. The benefits are obvious: I could finally get sieve filtering on my stupid corporate exchange mail server :-). From pixilla at macports.org Thu Sep 20 19:12:25 2012 From: pixilla at macports.org (Bradley Giesbrecht) Date: Thu, 20 Sep 2012 09:12:25 -0700 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. Is there a Johannes version 2? I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. http://johannes.sipsolutions.net/files/antispam.html Regards, Bradley Giesbrecht (pixilla) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2763 bytes Desc: not available URL: From jerry at seibercom.net Thu Sep 20 19:12:42 2012 From: jerry at seibercom.net (Jerry) Date: Thu, 20 Sep 2012 12:12:42 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3CDD.6@esiee.fr> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> Message-ID: <20120920121242.120102d6@scorpio> On Thu, 20 Sep 2012 17:57:17 +0200 Frank Bonnet articulated: > if you install from FreeBSD ports you need to enable > the MYSQL backend during configuration ( make config ) If you don't install Dovecot via the FreeBSD ports system, you are just asking for problems. As previously stated, do run "make config" and perhaps "make config-recursive" prior to actually attempting to build the port. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ben at indietorrent.org Thu Sep 20 19:33:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 12:33:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> Message-ID: <505B454D.6000402@indietorrent.org> On 9/20/2012 12:12 PM, Bradley Giesbrecht wrote: > > On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > >> Thanks again, Tom. After you set me straight on the various versions of >> the plug-in, I was able to synthesize the relevant documentation and >> make it all work on Ubuntu 12.04 LTS with Dovecot 2. >> >> The key was to understand the following: >> >> 1.) The Ubuntu repositories use the Johannes versions of the plug-in >> (for both Dovecot 1 and 2). >> >> Ubuntu 10.04 LTS (Dovecot 1) >> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html >> >> Ubuntu 12.04 LTS (Dovecot 2) >> http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html >> >> 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to >> the Eugene version, and that version's configuration directives differ >> from the Johannes version 2 directives. > > Is there a Johannes version 2? Thanks for your reply, Bradley. That seems to be the implication here: http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html Johannes is listed as an author, the package name is dovecot-antispam_2.0+20120225, and the manpage is nearly identical to that of version 1 at http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html . > I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > >> So, anyone attempting to follow >> the Wiki article on Ubuntu will need to adapt the directives accordingly >> (see documentation links above for equivalent directive names). >> >> 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki >> instructions lacks a shebang; until a proper shebang is added to the >> shell script, script execution will fail with "status 1". I added the >> standard shebang: >> >> #!/bin/sh >> >> Here are the configuration directives that work for me: >> >> plugin { >> antispam_debug_target = syslog >> antispam_backend = pipe >> antispam_signature = X-DSPAM-Signature >> antispam_signature_missing = error >> antispam_spam_pattern_ignorecase = SPAM;JUNK >> antispam_pipe_tmpdir = /tmp >> antispam_pipe_program = /usr/bin/sa-learn-pipe.sh >> antispam_pipe_program_spam_arg = --spam >> antispam_pipe_program_notspam_arg = --ham >> } >> >> My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. > http://johannes.sipsolutions.net/files/antispam.html This documentation seems to apply to the Dovecot 2 version; the antispam_backend directive doesn't seem to exist in version 1, at least according to the official Ubuntu package documentation (cited above). In fact, this appears to be the exact document that is used for the Ubuntu package for Dovecot 2 (also cited above). > > Regards, > Bradley Giesbrecht (pixilla) > From surreal at bsdmail.com Thu Sep 20 19:35:08 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 00:35:08 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <20120920121242.120102d6@scorpio> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> Message-ID: <505B45BC.8020109@bsdmail.com> On 9/21/2012 12:12 AM, Jerry wrote: > On Thu, 20 Sep 2012 17:57:17 +0200 > Frank Bonnet articulated: > >> if you install from FreeBSD ports you need to enable >> the MYSQL backend during configuration ( make config ) > If you don't install Dovecot via the FreeBSD ports system, you are just > asking for problems. As previously stated, do run "make config" and > perhaps "make config-recursive" prior to actually attempting to build > the port. > I did, MySQL is ticked when i build dovecot2 from /usr/ports/mail/dovecot2/ From ml at cibest.com Thu Sep 20 20:04:27 2012 From: ml at cibest.com (Cibest ML) Date: Thu, 20 Sep 2012 19:04:27 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> Message-ID: <505B4C9B.5080708@cibest.com> Hi Timo, I've finally solved the issue. As usual, thinking out of the box helped :) In fact, it was conf related but not where it was expected. There was an autocreate = Inbox & autosubscribe = Inbox in 90-plugin.conf Having this configuration seems to double count the Inbox quota during quota recalc (but not during normal IMAP operations like moving or receiving a message). Might be some kind of bug somewhere in recalc. The strange thing is that another more simple dovecot system we have (no nfs, all imap/smtp/filtering operations done on the server) also has this plugin conf but doesn't count quota twice. Anyway, after running a small script to recalc all the quota, everything is now fine. Thanks for you help -- Regards S?bastien On 19/09/2012 14:47, Timo Sirainen wrote: > On 19.9.2012, at 15.15, Cibest ML wrote: > >> There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). > No idea. INBOX is treated case-insensitively by IMAP protocol. > >> I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) > Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. > From daryl at isletech.net Thu Sep 20 21:42:05 2012 From: daryl at isletech.net (Daryl Richards) Date: Thu, 20 Sep 2012 14:42:05 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B45BC.8020109@bsdmail.com> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> Message-ID: <505B637D.4080901@isletech.net> On 12-09-20 12:35 PM, Surreal wrote: > On 9/21/2012 12:12 AM, Jerry wrote: >> On Thu, 20 Sep 2012 17:57:17 +0200 >> Frank Bonnet articulated: >> >>> if you install from FreeBSD ports you need to enable >>> the MYSQL backend during configuration ( make config ) >> If you don't install Dovecot via the FreeBSD ports system, you are just >> asking for problems. As previously stated, do run "make config" and >> perhaps "make config-recursive" prior to actually attempting to build >> the port. >> > I did, MySQL is ticked when i build dovecot2 from > /usr/ports/mail/dovecot2/ If you do dovecot --build-options, does it list mysql? root at mail:~# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql -- Daryl Richards Isle Technical Services Inc. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 20 23:17:41 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 20 Sep 2012 22:17:41 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <20120920201741.GA9646@daniel.localdomain> Timo Sirainen wrote: > doveadm multi [-A | -u wildcards] [ [...]] > > Thoughts? Would be nice if this also worked through doveadm director proxy and executes the command sequence on the correct backend host for each user. As command name I could also think of "doveadm sequence", which implies the commands being executed in serial order. Regards Daniel -- https://plus.google.com/103021802792276734820 From ben at indietorrent.org Fri Sep 21 00:09:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 17:09:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <505B85FD.4000604@indietorrent.org> On 9/20/2012 11:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > I'm using the following directives, but when I move mail from Inbox to > Junk, for example, I see no evidence that antispam is taking action. > > plugin { > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_mail_tmpdir = /tmp > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > } > > Dovecot writes nothing at all to syslog. No errors, no reference to > antispam. And the message is moved successfully (no failure/error notice > in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", > attempts to create a log file in /tmp, which works with Antispam 2. But > no such luck here; nothing is ever written to /tmp. > > Version 2 of the plug-in includes a verbose debugging mode, but it seems > that no such feature is present in version 1. > > I noticed also that version 1 seems not to require a back-end to be > specified explicitly (as in version 2). Is the back-end implied via the > configuration directives that are present? > > Respectfully, > > -Ben Just a follow-up to report that I finally have the Antispam plug-in (the Johannes version) -- with the mailtrain/pipe back-end -- working as expected in both of the following environments: - Ubuntu 10.04 with Dovecot 1.2.9 - Ubuntu 12.04 with Dovecot 2.0.19 In addition to the points that I mentioned in my previous note, a crucial step in the debugging process was to modify the "sa-learn-pipe.sh" script to produce more verbose output. Unlike Antispam 2, Antispam 1 lacks a verbose debugging mode, so when the plug-in does not work as expected, tracking-down the root cause can be much more difficult. Here's the shell script from http://wiki2.dovecot.org/Plugins/Antispam , with my modifications: http://pastebin.com/igTXCYQF After making these modifications to the shell script, I was able to determine the cause of the failure in Dovecot 1.2.9. In fact, I was incorrect earlier when I stated that version 2 was working; the same problem applied there, too. The "vmail" user, as whom the script is executed, lacks the permissions required to change the Bayes token database (this is a "good thing"). An excerpt from the custom log file: 29722-start (--spam) Calling (as user vmail) '/usr/bin/sa-learn --spam /tmp/sendmail-msg-29722.txt' bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 493. Learned tokens from 0 message(s) (1 message(s) examined) 29722-end The only means by which I was able to work around this issue -- without changing filesystem permissions, users, groups, or otherwise compromising system security -- was to migrate from the standard flat-file Bayes database format to MySQL. (The guide at http://www.starbridge.org/spip/spip.php?article15 had everything I needed, and it only took me a few minutes to implement.) After making that change, everything began working as expected; here's a sample custom log entry: 30565-start (--ham) Calling (as user vmail) '/usr/bin/sa-learn --ham /tmp/sendmail-msg-30565.txt' Learned tokens from 1 message(s) (1 message(s) examined) 30565-end In any event, this concludes the saga, and I must say, I'm thrilled to have it all working. Thanks again to everyone who contributed valuable input and pointed me in the right direction. And thanks to Johannes and Eugene (and everyone else who contributed to Antispam) for solving a major hurdle in Bayes DB training. -Ben From surreal at bsdmail.com Fri Sep 21 04:35:22 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 09:35:22 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B637D.4080901@isletech.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> <505B637D.4080901@isletech.net> Message-ID: <505BC45A.1010003@bsdmail.com> On 9/21/2012 2:42 AM, Daryl Richards wrote: > On 12-09-20 12:35 PM, Surreal wrote: >> On 9/21/2012 12:12 AM, Jerry wrote: >>> On Thu, 20 Sep 2012 17:57:17 +0200 >>> Frank Bonnet articulated: >>> >>>> if you install from FreeBSD ports you need to enable >>>> the MYSQL backend during configuration ( make config ) >>> If you don't install Dovecot via the FreeBSD ports system, you are just >>> asking for problems. As previously stated, do run "make config" and >>> perhaps "make config-recursive" prior to actually attempting to build >>> the port. >>> >> I did, MySQL is ticked when i build dovecot2 from >> /usr/ports/mail/dovecot2/ > If you do dovecot --build-options, does it list mysql? > > root at mail:~# dovecot --build-options > Build options: ioloop=kqueue notify=kqueue ipv6 openssl > io_block_size=8192 > Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw > SQL drivers: mysql > Passdb: checkpassword pam passwd passwd-file sql > Userdb: checkpassword nss passwd prefetch passwd-file sql > At first i have noticed that i dont have mysql when i did --build-options, so i tried to deinstall and reinstall dovecot2 port and now i have this also: hostname# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql But i still keep getting the same error hostname# service dovecot start doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver Starting dovecot. doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot Still the same output for doveconf -n hostname# doveconf -n # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = ren.wf auth_mechanisms = plain login auth_realms = ren.wf graphics.re disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } doveconf: Error: userdb is missing driver protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver From tss at iki.fi Fri Sep 21 08:28:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 21 Sep 2012 07:28:57 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <20120920201741.GA9646@daniel.localdomain> References: <20120920201741.GA9646@daniel.localdomain> Message-ID: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> On 20.9.2012, at 22.17, Daniel Parthey wrote: > Timo Sirainen wrote: >> doveadm multi [-A | -u wildcards] [ [...]] >> >> Thoughts? > > Would be nice if this also worked through doveadm director proxy > and executes the command sequence on the correct backend host for > each user. Yes, it's going to. > As command name I could also think of "doveadm sequence", which > implies the commands being executed in serial order. Hmm. Maybe. From alec at alec.pl Fri Sep 21 11:23:02 2012 From: alec at alec.pl (A.L.E.C) Date: Fri, 21 Sep 2012 10:23:02 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <505C23E6.2070606@alec.pl> On 09/20/2012 06:01 PM, Timo Sirainen wrote: > Thoughts? Any better name for the command than "multi"? How about 'execute' or 'exec'. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From tomas at blueboard.cz Fri Sep 21 12:32:37 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Fri, 21 Sep 2012 11:32:37 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? Message-ID: <505C3435.30601@blueboard.cz> Hello, I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or dovecot<>freebsd bug? I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over 1000 processes with kernel message: maxproc limit exceeded by uid 89 Could anybody help? Many thanks Tomas My system have following settings: FreeBSD 9.0 / AMD64 Dovecot 2.1.8 kern.maxproc: 12288 kern.maxfilesperproc: 36864 kern.maxprocperuid: 8192 no limit for uid 89: # limit cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited My dovecot.conf: # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 2048 default_process_limit = 2048 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /data/logfiles/dovecot/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_plugins = " quota" mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore sieve = ~/.sieve/dovecot.sieve sieve_dir = ~/.sieve } protocols = imap pop3 sieve service anvil { client_limit = 6147 } service auth { client_limit = 8192 unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 2048 service_count = 1 } service imap { client_limit = 1 process_limit = 2048 } service managesieve { process_limit = 2048 } service pop3-login { process_limit = 2048 service_count = 1 } service pop3 { client_limit = 1 process_limit = 2048 } shutdown_clients = no ssl_cert = object mailstore In-Reply-To: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> Message-ID: On 17 September 2012 13:57, Timo Sirainen wrote: > On 14.9.2012, at 18.16, Damien Churchill wrote: > >> On 14 September 2012 15:59, Timo Sirainen wrote: >>> On 14.9.2012, at 17.46, Damien Churchill wrote: >>> >>>> I was wondering what would be entailed in modifying dovecot 2.2 to >>>> support storing mail in an object store. I've seen a few mails dotted >>>> around in the ML history about supporting such a thing and seen it's >>>> basically dependant upon some changes in lib-storage to support >>>> writing messages without locking. Is this still the case? >>> >>> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >>> >> >> That's good to hear :) I've been evaluating RADOS as an object store, >> which is similar to S3. Although any distributed storage would be >> great. I'd be more than happy to help code it! > > I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. > >>> The generic idea is: >>> - only one server accesses one user simultaneously >>> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >>> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >>> - support high latency: asynchronous reads/writes. prefetch mail bodies. >> >> I'm assuming that the director would be used in order to distribute >> connections to the same server, so it's only within a local instance >> of dovecot you'd need to be aware of what currently has a connection >> open for that user? > > Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > Sounds good! I'll spend some time digging through the source code getting familiar. >> How are you planning on handling the situation where say node X dies >> and hasn't uploaded the latest index file? Would that result in >> missing messages from the mailbox when accessed by another node, or is >> the local index intended to be more of a write-through cache? > > No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) Part of me thinks making this configurable might be a good idea depending upon what the installation is trying to achieve. Since the recovery process will need to be implemented regardless allowing the user to configure dovecot to perform a write to both the local and object index when a flag or something else is modified. Another thought occurs to me, when using the LDA how will that be able to update the index upon delivery of a new message if another node is currently accessing the mailbox? From h.reindl at thelounge.net Fri Sep 21 12:52:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 11:52:50 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505C38F2.40402@thelounge.net> Am 21.09.2012 11:32, schrieb Tom?? Randa: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried > everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or > dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over > 1000 processes with kernel message: no idea about BSD but your config are a total of up to 10.240 PROCESSES one process has MUCH MORE than one file-handle i have ONE imap-login process with 572 file-handles your configuration eats up to 5 Mio. file-handles maybe you running out of OS ressources 1000 prcoesses are up to 500000 file handles for one service [root at mail:~]$ ps aux | grep imap-login | wc -l 2 [root at mail:~]$ lsof | grep imap-l | wc -l 572 ___________________ for imaplogin / pop3-login as example you do not need a PROCESS per connection service_count = 0 process_min_avail = 1 process_limit = 10 client_limit = 200 this can handle 2000 connections with up to 10 processes > service imap-login { > process_limit = 2048 > service_count = 1 > } > service imap { > client_limit = 1 > process_limit = 2048 > } > service managesieve { > process_limit = 2048 > } > service pop3-login { > process_limit = 2048 > service_count = 1 > } > service pop3 { > client_limit = 1 > process_limit = 2048 > } -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From trashcan at odo.in-berlin.de Fri Sep 21 14:48:45 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 21 Sep 2012 13:48:45 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: On 2012-09-21 10:23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. doveadm execall [...] Regards, Michael From matthias at couch-blog.de Fri Sep 21 16:09:06 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:09:06 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? Message-ID: <505C66F2.7060704@couch-blog.de> Hello! Could it be, that http://xi.rename-it.nl is down? Thanks, Matthias From h.reindl at thelounge.net Fri Sep 21 16:22:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:22:03 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C66F2.7060704@couch-blog.de> References: <505C66F2.7060704@couch-blog.de> Message-ID: <505C69FB.3040402@thelounge.net> Am 21.09.2012 15:09, schrieb Matthias: > Hello! > > Could it be, that http://xi.rename-it.nl is down? not from vienna but what the hell has this question to search on the dovecot-list? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Fri Sep 21 16:27:37 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 21 Sep 2012 15:27:37 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132737.GX9342@charite.de> * Reindl Harald : > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? Isn't that where the sieve extension is located at? http://xi.rename-it.nl/debian/dists/unstable-auto/ -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mailinglist at darac.org.uk Fri Sep 21 16:27:47 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 21 Sep 2012 14:27:47 +0100 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132747.GA23967@darac.org.uk> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? > The mentioned site is a host for Prebuilt Binaries: http://wiki2.dovecot.org/PrebuiltBinaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 21 16:40:51 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:40:51 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <20120921132747.GA23967@darac.org.uk> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> Message-ID: <505C6E63.6020608@thelounge.net> Am 21.09.2012 15:27, schrieb Darac Marjal: > On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >> >> >> Am 21.09.2012 15:09, schrieb Matthias: >>> Hello! >>> >>> Could it be, that http://xi.rename-it.nl is down? >> >> not from vienna >> >> but what the hell has this question to search >> on the dovecot-list? >> > > The mentioned site is a host for Prebuilt Binaries: > http://wiki2.dovecot.org/PrebuiltBinaries ok, this should have been mentioned because only few people understand "Welkom op de website van Rename-it, uw partner in systeembeheer en internetoplossingen." * most people using packages from their distribution * the rest is compiling from source * one or two others are using foreign binaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From sb at dod.no Fri Sep 21 16:53:29 2012 From: sb at dod.no (Steinar Bang) Date: Fri, 21 Sep 2012 15:53:29 +0200 Subject: [Dovecot] doveadm with multiple commands References: Message-ID: <87mx0jbhye.fsf@dod.no> >>>>> Timo Sirainen : > Thoughts? How about: a) Dropping into a command prompt where it's possible to give commands? b) Making it possible to feed commands through a pipe (ie reading lines from stdin)? From matthias at couch-blog.de Fri Sep 21 16:55:48 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:55:48 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C6E63.6020608@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> <505C6E63.6020608@thelounge.net> Message-ID: <505C71E4.6070105@couch-blog.de> Am 21.09.2012 15:40, schrieb Reindl Harald: > > Am 21.09.2012 15:27, schrieb Darac Marjal: >> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >>> >>> Am 21.09.2012 15:09, schrieb Matthias: >>>> Hello! >>>> >>>> Could it be, that http://xi.rename-it.nl is down? >>> not from vienna >>> >>> but what the hell has this question to search >>> on the dovecot-list? >>> >> The mentioned site is a host for Prebuilt Binaries: >> http://wiki2.dovecot.org/PrebuiltBinaries > ok, this should have been mentioned because only few > people understand "Welkom op de website van Rename-it, > uw partner in systeembeheer en internetoplossingen." > > * most people using packages from their distribution > * the rest is compiling from source > * one or two others are using foreign binaries > Oh now all is great, thanks for small help ;) I test a little bit with a small sieve skript, to filter my mails on my mailserver with dovecot 2.1. Now it just works. Sorry if the question was on the wrong mailinglist. From alessio at skye.it Fri Sep 21 17:13:15 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 21 Sep 2012 16:13:15 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <1347680376.5717.61.camel@maclinux> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <1347680376.5717.61.camel@maclinux> Message-ID: <505C75FB.4080006@skye.it> Il 15/09/2012 05:39, Jeff Gustafson ha scritto: > On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: >> I've a whole new design for it and I was planning on implementing it >> for v2.2. Do you want to help coding it? :) Which storage would you >> want to use? >> >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and >> accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split >> brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > With this system, would the read/write ultimately go to a normal OS > file function? If it is a file function, could this be used with a > system like glusterfs, ceph, etc? The other option would be to write it > against a object store client library and bypass the normal file > functions. > > ...Jeff > Also other users are talking about Ceph and Dovecot http://www.mail-archive.com/ceph-devel at vger.kernel.org/msg07345.html -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stan at hardwarefreak.com Sat Sep 22 00:16:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 21 Sep 2012 16:16:50 -0500 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505CD942.7000104@hardwarefreak.com> On 9/21/2012 4:32 AM, Tom?? Randa wrote: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. What is your value for kern.maxusers? Did you try increasing it? Note in the 2nd paragraph below the relationship between kern.maxusers and process limit. From what you describe it would seem you have a process limit of 1044, thus a kern.maxusers value of 64. Considering your manual setting of 8192 processes is apparently being ignored, it would seem the kern.maxusers value is causing it to be overridden. From: http://www.pl.freebsd.org/doc/handbook/configtuning-kernel-limits.html As of FreeBSD 4.5, kern.maxusers is automatically sized at boot based on the amount of memory available in the system, and may be determined at run-time by inspecting the value of the read-only kern.maxusers sysctl. Some sites will require larger or smaller values of kern.maxusers and may set it as a loader tunable; values of 64, 128, and 256 are not uncommon. We do not recommend going above 256 unless you need a huge number of file descriptors; many of the tunable values set to their defaults by kern.maxusers may be individually overridden at boot-time or run-time in /boot/loader.conf (see the loader.conf(5) man page or the /boot/defaults/loader.conf file for some hints) or as described elsewhere in this document. Systems older than FreeBSD 4.4 must set this value via the kernel config(8) option maxusers instead. In older releases, the system will auto-tune maxusers for you if you explicitly set it to 0[1]. When setting this option, you will want to set maxusers to at least 4, especially if you are using the X Window System or compiling software. The reason is that the most important table set by maxusers is the maximum number of processes, which is set to 20 + 16 * maxusers, so if you set maxusers to 1, then you can only have 36 simultaneous processes, including the 18 or so that the system starts up at boot time and the 15 or so you will probably create when you start the X Window System. Even a simple task like reading a manual page will start up nine processes to filter, decompress, and view it. Setting maxusers to 64 will allow you to have up to 1044 simultaneous processes, which should be enough for nearly all uses. If, however, you see the dreaded proc table full error when trying to start another program, or are running a server with a large number of simultaneous users (like ftp.FreeBSD.org), you can always increase the number and rebuild. -- Stan From durket at highwire.stanford.edu Sat Sep 22 00:54:02 2012 From: durket at highwire.stanford.edu (Michael Durket) Date: Fri, 21 Sep 2012 14:54:02 -0700 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? Message-ID: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? From robert at schetterer.org Sat Sep 22 08:46:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 07:46:15 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> Message-ID: <505D50A7.3090003@schetterer.org> Am 21.09.2012 23:54, schrieb Michael Durket: > I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): > > Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) > > Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? > you should upgrade to recent stable version 2.1.x -- Best Regards MfG Robert Schetterer From tlhackque at yahoo.com Sat Sep 22 15:04:22 2012 From: tlhackque at yahoo.com (tlhackque) Date: Sat, 22 Sep 2012 08:04:22 -0400 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" Message-ID: <505DA946.90409@yahoo.com> Dovecot 2.1.10 Client = Thunderbird. Local disks. mbox format. Attempted to rename a folder, failed with: CANNOT Renaming not supported across conflicting directory permissions I don't understand why this should happen. o I was renaming within a directory - the attempt was to rename "Domain Names &- SSL Certificates" to "Domain Names". No directory was specified o The reason the directory has g=s is for dovecot - so that the 'mail' group (12) will stick to any files created. I was getting more understandable permission failures before I added it. Since 's' doesn't apply to files, I don't see why this should be viewed as a conflict. o If I chmod g-s the directory, the command succeeds. But then I'm back to file ownership problems... drwxrws--- 4 4503 12 4096 Sep 22 06:17 F/N/ -rw-rw---- 1 4503 12 81530 Sep 22 06:17 F/N/Domain Names &- SSL Certificates Sequence from TB's trace (crlfs for clarity): 1140[11ebcf00]: 121e8c00:imap.example.com:A:SendData: 66 rename "F/N/Domain Names &- SSL Certificates" "F/N/Domain Names" 1140[11ebcf00]: ReadNextLine [stream=12135a28 nb=80 needmore=0] 1140[11ebcf00]: 121e8c00:imap.example.com:A:CreateNewLineFromSocket: 66 NO [CANNOT] Renaming not supported across conflicting directory permissions 9592[c7d6840]: ReadNextLine [stream=ab687e8 nb=22 needmore=0] Config: # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.22.14-72.fc6 i686 Fedora Core release 6 (Zod) first_valid_gid = 4000 first_valid_uid = 4000 hostname = smtp.example.com login_greeting = Dovecot ready. Unauthorized access is prohibited. mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { driver = pam } service imap-login { inet_listener imap { address = imap.v4.example.com imap.v6.example.com port = 143 } inet_listener imaps { address = imap.v4.example.com imap.v6.example.com port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { address = pop.v4.example.com pop.v6.example.com port = 110 } inet_listener pop3s { address = pop.v4.example.com pop.v6.example.com port = 995 ssl = yes } } ssl_ca = Hi, running in problems with KMail I decided to switch to dovecot-imap & thunderbird on my kubuntu 12.04 64 bit (so I can read my imap-folder-mails with KMail again if it gets more stable) What I did: - I wrote a python script to convert my KMail mails folder to a thunderbird mailfolder (directory tree containing mbox files and *.sbd folders) - I created a fake pop mail account in thunderbird and imported the whole mail-structure (i just copied it) - I create a local imap account ( so I have a ~/Maildir dovecot folder in my user dir) - In /etc/dovecot: 1. /etc/dovecot/10-mail.conf (line 10) I set: mail_location = maildir:~/Maildir:LAYOUT=fs 2. /etc/dovecot/20-imap.conf (line 57) I set: imap_client_workarounds = tb-extra-mailbox-sep Now I want to copy the whole mailstructure from my imported fake pop mailfolder to the dovecot imap folder. What happens: Lets say my source structure is mailfolder1 + mail1 + mail2 + subMailFolder2 + mail1-in-subMailFolder2 + mail2-in-subMailFolder2 + subSubMailFolder3 + mail1-in-subMailFolder3 + mail2-in-subMailFolder3 I thunderbird (version 15) I drag and drop my mailFolder1 to the imap folder and I get: mailfolder1 + mail1 + mail2 + subMailFolder2 + subSubMailFolder3 as you See only the mails of the *TOP* folder are copied. All subfolders are created but they contain NO MAILS. Did somebody experience the same problem??? Otherwise I will write a python script to do that (I dont hope to need to write a mailclient in python ;-) ) Any hint is greatly welcome. Anton From me at junc.org Sat Sep 22 16:57:54 2012 From: me at junc.org (Benny Pedersen) Date: Sat, 22 Sep 2012 15:57:54 +0200 Subject: [Dovecot] =?utf-8?q?Dovecot_1=2E2=2E11_panic_in_mail-transaction-?= =?utf-8?q?log-view=3F?= In-Reply-To: <505D50A7.3090003@schetterer.org> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: Robert Schetterer skrev den 2012-09-22 07:46: > you should upgrade to recent stable version 2.1.x and this is a simple change from 1.x to 2.x :( i have never a panic in 1.x here From robert at schetterer.org Sat Sep 22 17:03:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 16:03:12 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: <505DC520.9050505@schetterer.org> Am 22.09.2012 15:57, schrieb Benny Pedersen: > Robert Schetterer skrev den 2012-09-22 07:46: > >> you should upgrade to recent stable version 2.1.x > > and this is a simple change from 1.x to 2.x :( depends on how complex your setup is youre should really try upgrade http://wiki.dovecot.org/Upgrading/1.2 http://wiki2.dovecot.org/Upgrading/2.0 > > i have never a panic in 1.x here > yeah, but i guess active bug supported are only recent versions, however wait till Timos answers > > > -- Best Regards MfG Robert Schetterer From csch at kiez.net Sat Sep 22 19:11:06 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 18:11:06 +0200 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 Message-ID: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Ok, here's a toughie: Out of a whim (and because of the bad weather) I today decided to upgrade my completely functioning 2.0.15 installation on my Mac OS X 10.6.7 system. NB: It's not a Mac OS X "Server", as sold by Apple - I have compiled my dovecots myself for quite a while. Anyhow: At first everything appeared to work fine after the upgrade. Up until I created a new user and -while testing- SELECTed his INBOX. The SELECT was "stuck". Though the process seemed to be alive I could only get rid of it with a "kill -9 ". The same happened when selecting a newly created mailbox on any of the other (otherwise functioning) accounts, so the problem had nothing to do with the new account, but rather with the fact, that it's INBOX was empty. Using dtruss and gdb I found out, that the dovecot process was trying to obtain a GUID and in the course of doing so invoked gethostbyname(), which intern caused a lot of "mach message" handling and somewhere, deep down there, the process was stuck. Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: sudo /usr/libexec/StartupItemContext dovecot (again: all this was working fine under 2.0.15) Now with 2.1.10, when I manually invoke dovecot with just sudo dovecot Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. Now -with all that said- here's my question: What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? It appears, that the imap process no longer "inherits" the StartupItemContext from the main process, so some change between 2.0.15 and 2.1.10 must have broken it ... Any help is highly appreciated - Clemens PS: I google'd around a lot and searched the mailing-lists, of course. I only found a post of someone who ran into the same/similar problem dating back to Feb 21st 2012 under the subject "dovecot freezes when trying to get mail from maildir with mail", but it was quickly dismissed without ever getting resolved and that was that. PS2: I intentionally didn't include any configs with this mail as they seem to be irrelevant, but of course I can generate the necessary output if needed. From tss at iki.fi Sat Sep 22 19:50:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:50:27 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> On 21.9.2012, at 11.23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. From tss at iki.fi Sat Sep 22 19:52:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:52:39 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <87mx0jbhye.fsf@dod.no> References: <87mx0jbhye.fsf@dod.no> Message-ID: On 21.9.2012, at 16.53, Steinar Bang wrote: >>>>>> Timo Sirainen : > >> Thoughts? > > How about: > a) Dropping into a command prompt where it's possible to give commands? > b) Making it possible to feed commands through a pipe (ie reading lines > from stdin)? Both of these would require adding some kind of new command parameter parsing. Currently shell does all the "multi word string" parsing and escaping and such. My original thought was also to be able to read commands from stdin, but I'd rather not add such special command parser, at least not yet.. From jcblanco at fi.upm.es Sat Sep 22 20:29:46 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Sat, 22 Sep 2012 19:29:46 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <505A09D0.6040607@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> <505A09D0.6040607@rename-it.nl> Message-ID: <505DF58A.3020403@fi.upm.es> Sorry I've sent a config.log but get retained for approval. This is the beginning of the file: This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by Pigeonhole configure 0.3.3, which was generated by GNU Autoconf 2.59. Invocation command line was $ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --target=x86_64-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-dovecot=/home/jc/rpmbuild/BUILD/dovecot-2.1.10 --with-managesieve=yes --enable-header-install=yes INSTALL_DATA=install -c -p -m644 ## --------- ## ## Platform. ## ## --------- ## hostname = testv.fi.upm.es uname -m = x86_64 uname -r = 2.6.18-308.13.1.el5debug uname -s = Linux uname -v = #1 SMP Tue Aug 21 18:34:48 EDT 2012 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = x86_64 /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown Regards Juan C. Blanco On 19/09/2012 20:07, Stephan Bosch wrote: > On 9/19/2012 6:22 PM, Juan C. Blanco wrote: >> On 18/09/2012 20:49, Stephan Bosch wrote: >>> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>>> It seems that the error is related to the ld.gold fix, If I revert the >>>> c52a0c561311 patch the error is not produced >>> >>> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >>> but I forgot the Dovecot v2.1 tree. Fixed now: >>> >>> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > D'oh, this is the v2.2 change. The URL I meant is this one: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >>> >>> This only happens when you compile against installed Dovecot headers, >>> people using --with-dovecot= (like me) would not have seen this. >>> >>> I'll push out a new release once 2.1.10 comes out. I'm being told that >>> is imminent. >> >> Sorry, but using the new 0.3.3 version I'm getting the same error and >> I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in >> configure, not the installed headers. If needed I can send you the >> configure invocation command. > > Yes, that could be helpful. > > Regards, > > Stephan. > From csch at kiez.net Sat Sep 22 22:51:22 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 21:51:22 +0200 Subject: [Dovecot] Update: Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 (and 2.0.21) References: <692EAFE7-E424-4F95-8DFA-C963B4957E11@kiez.net> Message-ID: Update: I tried with 2.0.21 and this also works just fine. So it must be something which came in with 2.1.x PS: Where is the documentaion for 2.1.x - i.e. for all the nice additions Timo made? The Website "only" has 2.0.x, as far as I can tell? (might be wrong here - hadn't touched the whole thing for a while, as my "civil life" had occupied me :-) Greetings, Clemens From joe at netmusician.org Sat Sep 22 22:52:42 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:52:42 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E170A.5090800@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sat Sep 22 22:57:03 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:57:03 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E180F.5060407@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. From mr88talent at gmail.com Sun Sep 23 04:00:21 2012 From: mr88talent at gmail.com (Gary V) Date: Sat, 22 Sep 2012 19:00:21 -0600 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On Sat, Sep 22, 2012 at 1:57 PM, Joe Auty wrote: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I can > try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp >> 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple SSL > certs. May not be related at all, but when I see segfaults on Debian, the first thing I try is to switch to bash as the default system shell: dpkg-reconfigure dash Answer: Use dash as the default system shell (/bin/sh)? -- Gary V From joe at netmusician.org Sun Sep 23 07:58:54 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 00:58:54 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505E970E.1040803@netmusician.org> > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sun Sep 23 08:03:28 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 01:03:28 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E970E.1040803@netmusician.org> References: <505E180F.5060407@netmusician.org> <505E970E.1040803@netmusician.org> Message-ID: <505E9820.5070007@netmusician.org> > > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. P.S. sorry for the duplicate message, it seems to have something to do with attaching images to messages. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ > From robert at schetterer.org Sun Sep 23 08:44:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 07:44:36 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: <505EA1C4.2000808@schetterer.org> Am 22.09.2012 21:57, schrieb Joe Auty: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I > can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 >> sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple > SSL certs. show your dovecot conf -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 09:00:10 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 02:00:10 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA1C4.2000808@schetterer.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> Message-ID: <505EA56A.6010102@netmusician.org> > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf Thanks! http://pastebin.com/gVRrCjyK I took what you said literally, please let me know if you'd like to see any of the other Debian config files. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org From weber at zackbummfertig.de Sun Sep 23 10:56:52 2012 From: weber at zackbummfertig.de (weber at zackbummfertig.de) Date: Sun, 23 Sep 2012 09:56:52 +0200 Subject: [Dovecot] Dovecot Clustering with dsync over ssh Message-ID: Hello List, i only found an old post of tiemo where he announced that he will do an wiki article how to setup dovecot cluster with dsync over ssh. anyone knows if this artcile exists and where? i would like to setup a dovecot cluster with 2 machines. thanks for help marko From dovecot-list at mohtex.net Sun Sep 23 12:07:29 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Sun, 23 Sep 2012 16:07:29 +0700 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505ED151.8090807@mohtex.net> Joe Auty wrote the following on 23.09.2012 13:00: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > With "show your dovecot conf" Robert means show us the output of "doveconf -n" From joe at netmusician.org Sun Sep 23 12:16:15 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 05:16:15 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED151.8090807@mohtex.net> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> Message-ID: <505ED35F.1020402@netmusician.org> > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: From tss at iki.fi Sun Sep 23 12:58:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 23 Sep 2012 12:58:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On 22.9.2012, at 22.57, Joe Auty wrote: > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work: service imap-login { executable = imap-login -D } Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full From robert at schetterer.org Sun Sep 23 13:27:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 12:27:15 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505EE403.8060209@schetterer.org> Am 23.09.2012 08:00, schrieb Joe Auty: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK fine, but what i/we like to see is the result of dovecot -n on the list, and pleady avoid mail your jpg only to the list ( what ever this is ) > > I took what you said literally, please let me know if you'd like to see > any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I >> suspect this might be causing performance issues. Any suggestions what >> I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 13:44:23 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 06:44:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED35F.1020402@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> <505ED35F.1020402@netmusician.org> Message-ID: <505EE807.40903@netmusician.org> > > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From joe at netmusician.org Sun Sep 23 14:05:36 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 07:05:36 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505EED00.6090109@netmusician.org> > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full I hope I'm doing this correctly! # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...(no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f789cf5e018 in login_binary_run () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #5 0x0000000000402459 in ?? () No symbol table info available. #6 0x00007fff8a9c65f8 in ?? () No symbol table info available. #7 0x000000000000001c in ?? () No symbol table info available. #8 0x0000000000000002 in ?? () No symbol table info available. #9 0x00007fff8a9c7e6a in ?? () No symbol table info available. #10 0x00007fff8a9c7e7d in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Sun Sep 23 16:51:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 09:51:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505F13DB.1060400@Media-Brokers.com> On 2012-09-23 2:00 AM, Joe Auty wrote: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK Please don't make it harder for others to help you... Just paste the output in the body of the email. Some (many?) people won't click on links like that from people they don't know... -- Best regards, Charles From stephan at rename-it.nl Sun Sep 23 20:31:27 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:31:27 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <20120920081126.0627cd71@tomh> References: <20120920081126.0627cd71@tomh> Message-ID: <505F476F.5090305@rename-it.nl> On 9/20/2012 2:11 PM, Tom Horsley wrote: > A slightly off topic question, but maybe someone here has seen > something like this: > > I want a program that operates entirely on the client side of > an IMAP server, and implements the sieve filtering language > by doing the necessary client side operations to move mail > on the IMAP server, expunge mail, etc. > > It would also act as a proxy between my real mail client and > the IMAP server so I'd never see any mail that hasn't been > filtered first. > > The benefits are obvious: I could finally get sieve filtering > on my stupid corporate exchange mail server :-). Interesting idea. I've been experimenting a bit with the sieve-filter command line tool and the Dovecot imapc mail storage backend. This does not entirely match your proposed solution, since it'll have to operate as a cron job, but at least it is a start. I've configured my experiment as follows: => I built a simple dovecot.conf protocols = none listen = *, :: mail_location = imapc:~/imapc imapc_host = your.host.tld imapc_port = 143 imapc_user = username imapc_password = imapc_ssl = starttls imapc_ssl_verify = no => I tested whether IMAP connection works: stephan at klara:~/test$ /usr/lib/dovecot/imap -c ./dovecot.conf * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in as stephan 2342 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8274 EXISTS * 0 RECENT * OK [UIDVALIDITY 1348169448] UIDs valid * OK [UIDNEXT 8275] Predicted next UID * OK [NOMODSEQ] No permanent modsequences 2342 OK [READ-WRITE] Select completed. => I built a Sieve script called test.sieve: require "fileinto"; require "mailbox"; # Just moves mail elsewhere fileinto :create "Target"; => And finally tested operation using sieve-filter: sieve-filter -v -e -W -c dovecot.conf test.sieve INBOX In the process a few problems were encountered and solved. This means that this little experiment will only work with the latest revisions of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective Mercurial repositories. The conclusion so far is that, in essence, what you'd like to do is possible. However, to make Sieve filtering look like it is being run at delivery rather than at regular intervals in the background, we will need to build some sort of IMAP plugin that monitors INBOX for new messages and passes them through a Sieve filter before the user's proxied IMAP client can see them. I'll have to investigate how much work implementing this would be. Regards, Stephan. From stephan at rename-it.nl Sun Sep 23 20:43:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:43:41 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <505F476F.5090305@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> Message-ID: <505F4A4D.6060304@rename-it.nl> On 9/23/2012 7:31 PM, Stephan Bosch wrote: > On 9/20/2012 2:11 PM, Tom Horsley wrote: >> I want a program that operates entirely on the client side of >> an IMAP server, and implements the sieve filtering language >> by doing the necessary client side operations to move mail >> on the IMAP server, expunge mail, etc. >> > Interesting idea. I've been experimenting a bit with the sieve-filter > command line tool and the Dovecot imapc mail storage backend. This > does not entirely match your proposed solution, since it'll have to > operate as a cron job, but at least it is a start. [...] > In the process a few problems were encountered and solved. This means > that this little experiment will only work with the latest revisions > of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective > Mercurial repositories. > > The conclusion so far is that, in essence, what you'd like to do is > possible. However, to make Sieve filtering look like it is being run > at delivery rather than at regular intervals in the background, we > will need to build some sort of IMAP plugin that monitors INBOX for > new messages and passes them through a Sieve filter before the user's > proxied IMAP client can see them. I'll have to investigate how much > work implementing this would be. I should explain explain in a little more detail how this works. Basically, I use the sieve-filter tool to filter mail that is already in a mailbox. Only, the mailbox is a bit special: it is an imapc (IMAP Client) mailbox that proxies to a remote mailbox. So, on our local machine the INBOX actually maps to INBOX on the remote server, rather than some local store. For the sieve-filter tool this makes no difference. This way, the sieve-filter can also be used to filter mail on a remote server. Regards, Stephan. From me at junc.org Sun Sep 23 20:47:17 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 23 Sep 2012 19:47:17 +0200 Subject: [Dovecot] =?utf-8?q?Sieve_as_mail_client=3F?= In-Reply-To: <505F4A4D.6060304@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> Message-ID: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Stephan Bosch skrev den 23-09-2012 19:43: > This way, the sieve-filter can also be used to filter mail > on a remote server. is it gpl ?, anyway pretty cool From stephan at rename-it.nl Sun Sep 23 21:44:36 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 20:44:36 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Message-ID: <505F5894.8040809@rename-it.nl> On 9/23/2012 7:47 PM, Benny Pedersen wrote: > Stephan Bosch skrev den 23-09-2012 19:43: >> This way, the sieve-filter can also be used to filter mail >> on a remote server. > > is it gpl ? The sieve-filter tool is part of Pigeonhole. Both Dovecot and Pigeonhole are mostly LGPLv2.1. Check the COPYING files for details. Regards, Stephan. From joe at netmusician.org Sun Sep 23 22:45:33 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 15:45:33 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F13DB.1060400@Media-Brokers.com> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> Message-ID: <505F66DD.70603@netmusician.org> > > Charles Marcus > September 23, 2012 9:51 AM > > > Please don't make it harder for others to help you... > > Just paste the output in the body of the email. Some (many?) people > won't click on links like that from people they don't know... My apologies, I figured at the time that it would be better to not make these emails crazy long, but I guess they can be just as easily snipped... Here is my conf: ## Dovecot configuration file # If you're in a hurry, see http://wiki2.dovecot.org/QuickConfiguration # "doveconf -n" command gives a clean output of the changed settings. Use it # instead of copy&pasting files when posting to the Dovecot mailing list. # '#' character and everything after it is treated as comments. Extra spaces # and tabs are ignored. If you want to use either of these explicitly, put the # value inside quotes, eg.: key = "# char and trailing whitespace " # Default values are shown for each setting, it's not required to uncomment # those. These are exceptions to this though: No sections (e.g. namespace {}) # or plugin settings are added by default, they're listed only as examples. # Paths are also just examples with the real defaults being based on configure # options. The paths listed here are for configure --prefix=/usr # --sysconfdir=/etc --localstatedir=/var # Enable installed protocols !include_try /usr/share/dovecot/protocols.d/*.protocol # A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, :: # Base directory where to store runtime data. #base_dir = /var/run/dovecot/ # Name of this instance. In multi-instance setup doveadm and other commands # can use -i to select which instance is used (an alternative # to -c ). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot # Greeting message for clients. #login_greeting = Dovecot ready. # Space separated list of trusted network ranges. Connections from these # IPs are allowed to override their IP addresses and ports (for logging and # for authentication checks). disable_plaintext_auth is also ignored for # these networks. Typically you'd specify your IMAP proxy servers here. #login_trusted_networks = # Sepace separated list of login access check sockets (e.g. tcpwrap) #login_access_sockets = # With proxy_maybe=yes if proxy destination matches any of these IPs, don't do # proxying. This isn't necessary normally, but may be useful if the destination # IP is e.g. a load balancer's IP. #auth_proxy_self = # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). #verbose_proctitle = no verbose_proctitle = yes # Should all processes be killed when Dovecot master process shuts down. # Setting this to "no" means that Dovecot can be upgraded without # forcing existing client connections to close (although that could also be # a problem if the upgrade is e.g. because of a security fix). #shutdown_clients = yes # If non-zero, run mail commands via this many connections to doveadm server, # instead of running them directly in the same process. #doveadm_worker_count = 0 # UNIX socket or host:port used for connecting to doveadm server #doveadm_socket_path = doveadm-server # Space separated list of environment variables that are preserved on Dovecot # startup and passed down to all of its child processes. You can also give # key=value pairs to always set specific settings. #import_environment = TZ ## ## Dictionary server settings ## # Dictionary can be used to store key=value lists. This is used by several # plugins. The dictionary can be accessed either directly or though a # dictionary server. The following dict block maps dictionary names to URIs # when the server is used. These can then be referenced using URIs in format # "proxy::". dict { #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext } # Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf # A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf > > > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Mon Sep 24 01:42:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 18:42:09 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F66DD.70603@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> <505F66DD.70603@netmusician.org> Message-ID: <505F9041.6080700@Media-Brokers.com> On 2012-09-23 3:45 PM, Joe Auty wrote: >> >> Charles Marcus >> September 23, 2012 9:51 AM >> >> >> Please don't make it harder for others to help you... >> >> Just paste the output in the body of the email. Some (many?) people >> won't click on links like that from people they don't know... > > My apologies, I figured at the time that it would be better to not > make these emails crazy long, but I guess they can be just as easily > snipped... Here is my conf: > > ## Dovecot configuration file No, you got it right the second time... When I said 'output', I mean the output of doveconf -n... which you did in a follow up email... -- Best regards, Charles From amateo at um.es Mon Sep 24 09:17:11 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 24 Sep 2012 08:17:11 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505FFAE7.3070905@um.es> El 21/09/12 11:32, Tom?? Randa escribi?: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. Could it be > some dovecot bug or dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap > clients in one process, but still I am over 1000 processes with kernel > message: > > maxproc limit exceeded by uid 89 > > > Could anybody help? Many thanks Tomas > Hi, I don't know BSD, but we had a similar problems with linux, when we reached 1024 processes, no more processes were created and we had errors like "imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument". If this is your same case, you could look for more info at http://www.dovecot.org/list/dovecot/2012-July/067014.html -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From manu at netbsd.org Mon Sep 24 10:39:54 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Mon, 24 Sep 2012 07:39:54 +0000 Subject: [Dovecot] dsync Message-ID: <20120924073954.GB16335@homeworld.netbsd.org> Hi Testing dsync, things go wrong: doveadm sync -u user remote:root at mail2.example.net dsync-local(user): Error: Mailboxes don't have unique GUIDs: 72e3be2c6f203b50883c000044af56a8 is shared by RT and RT_72e3be2c6f203b50883c000044af56a8 Obviously RT_72e3be2c6f203b50883c000044af56a8 is an outdated copy of RT But .mailboxlist does not list that mailbox. Is there a trick to make sure dsync only use valid mailboxes? I have this in dovecot.conf mail_location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=/mail/indexes/%u:SUBSCRIPTI ONS=../.mailboxlist Another problem, that may or may not be related: dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 60298748 dsync-local(user): Error: Failed to sync mailbox RT: Timeout while waiting for lock dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 63587421 dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files I also get this: dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files dsync-local(user): Error: proxy client timed out (waiting for MSG-GET message from remote) And this: dsync-local(user): Error: read() from worker server failed: EOF And generally speaking ,how good is dsync? is it usabel in production? This is on dovecot 2.1.7 -- Emmanuel Dreyfus manu at netbsd.org From dovecot at lists.wgwh.ch Mon Sep 24 11:49:22 2012 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Mon, 24 Sep 2012 10:49:22 +0200 Subject: [Dovecot] 2.1.10 imapc assert crash report Message-ID: <20120924104922.211d9ca3@boscos> Hi Timo I have a simple imapc gmail proxy test setup which works fine on 2.1.9, but crashes on 2.1.10 # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.4.7-1-ARCH x86_64 auth_mechanisms = plain login imapc_host = imap.gmail.com imapc_port = 993 imapc_ssl = imaps imapc_ssl_ca_dir = /etc/ssl/certs listen = 127.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=imap.gmail.com port=993 ssl=imaps default_fields = userdb_imapc_user=%u userdb_imapc_password=%w userdb_imapc_ssl=imaps userdb_imapc_port=993 driver = imap } protocols = imap ssl = no userdb { driver = prefetch } Log: Sep 24 10:21:58 codemonkey dovecot: master: Dovecot v2.1.10 starting up Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Sep 24 10:22:12 codemonkey dovecot: auth: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x453aa) [0x7f8d5ce963aa] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x453ee) [0x7f8d5ce963ee] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8d5ce6abd3] -> /usr/local/lib/dovecot/auth/libauthdb_imap.so(+0x977c) [0x7f8d5be3677c] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3618e) [0x7f8d5ce8718e] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f8d5cea3006] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f8d5cea3df7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f8d5cea2b48] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8d5ce8eb93] -> dovecot/auth(main+0x2ff) [0x40ad4f] -> /lib/libc.so.6(__libc_start_main+0xf5) [0x7f8d5c27d725] -> dovecot/auth() [0x40af61] Sep 24 10:22:12 codemonkey dovecot: auth: Fatal: master: service(auth): child 24008 killed with signal 6 (core not dumped) Sep 24 10:22:12 codemonkey dovecot: imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=24007, EOF) Oli -- message transmitted on 100% recycled electrons From tss at iki.fi Mon Sep 24 14:11:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:11:24 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: On 21.9.2012, at 8.28, Timo Sirainen wrote: >> Timo Sirainen wrote: >>> doveadm multi [-A | -u wildcards] [ [...]] >>> >>> Thoughts? > >> As command name I could also think of "doveadm sequence", which >> implies the commands being executed in serial order. > > Hmm. Maybe. "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. From tss at iki.fi Mon Sep 24 14:17:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:17:16 +0300 Subject: [Dovecot] 2.1.10 imapc assert crash report In-Reply-To: <20120924104922.211d9ca3@boscos> References: <20120924104922.211d9ca3@boscos> Message-ID: <0734901B-7805-42C5-9039-F0D6C2EC566B@iki.fi> On 24.9.2012, at 11.49, Oli Schacher wrote: > I have a simple imapc gmail proxy test setup which works fine on 2.1.9, > but crashes on 2.1.10 > Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/fd863826c892 http://hg.dovecot.org/dovecot-2.1/rev/17a8f15beb8c From Ralf.Hildebrandt at charite.de Mon Sep 24 14:27:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:27:06 +0200 Subject: [Dovecot] Logging question regarding delete actions Message-ID: <20120924112706.GS4043@charite.de> A user is logged in via imap from multiple devices. The log has this: Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 How can I find out WHICH CLIENT caused the deletion? Same issue: Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 How can I be sure that the log entry from 10:06:17 "belongs" to the log entry from "09:36:05"? Also, what is the meaning of the "session="? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:36:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:36:45 +0300 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <20120924112706.GS4043@charite.de> References: <20120924112706.GS4043@charite.de> Message-ID: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> On 24.9.2012, at 14.27, Ralf Hildebrandt wrote: > A user is logged in via imap from multiple devices. > The log has this: > > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 > > How can I find out WHICH CLIENT caused the deletion? Change mail_log_prefix to include %{session} (and maybe %r for IP). > Same issue: > > Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= > Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 > > How can I be sure that the log entry from 10:06:17 "belongs" to the > log entry from "09:36:05"? Also, what is the meaning of the > "session="? This is also solved with mail_log_prefix change. The session's idea is exactly to match the same session's log messages together. It's a string guaranteed to be unique for the next .. was it 7 years or so. From alec at alec.pl Mon Sep 24 14:44:29 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 24 Sep 2012 13:44:29 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> Message-ID: <5060479D.3060609@alec.pl> On 09/22/2012 06:50 PM, Timo Sirainen wrote: > On 21.9.2012, at 11.23, A.L.E.C wrote: > >> On 09/20/2012 06:01 PM, Timo Sirainen wrote: >>> Thoughts? Any better name for the command than "multi"? >> >> How about 'execute' or 'exec'. > > v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. next is "run" or "pipe", but what if you create global separator option and detect multi-command syntax usage automatically without a keyword? Syntax for doveadm would be doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command [command_options] [command_arguments] [separator command [command_options] [command_arguments] [...]] and example doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Ralf.Hildebrandt at charite.de Mon Sep 24 14:56:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:56:45 +0200 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> References: <20120924112706.GS4043@charite.de> <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> Message-ID: <20120924115645.GY4043@charite.de> * Timo Sirainen : > This is also solved with mail_log_prefix change. The session's idea is > exactly to match the same session's log messages together. It's a > string guaranteed to be unique for the next .. was it 7 years or so. Thanks. I changed the mail_log_prefix from mail_log_prefix = "%s(%u): " to mail_log_prefix = "%s(%u) %{session}: " -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:58:53 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5060479D.3060609@alec.pl> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> Message-ID: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> On 24.9.2012, at 14.44, A.L.E.C wrote: > next is "run" or "pipe", but what if you create global separator option > and detect multi-command syntax usage automatically without a keyword? > > Syntax for doveadm would be > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > [command_options] [command_arguments] [separator command > [command_options] [command_arguments] [...]] > > and example > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge Hmm. Yes, that might work. Although it would have to be: doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge because both -A and -s are mail command specific parameters, which won't work for non-mail commands. Hmm. This reminds me also that it would be possible with some extra work to do some command interaction. IMAP supports saving search results, which can later be accessed with $ parameter. So this could be made to work: doveadm search -s : from foo : fetch text \$ : expunge \$ From chevalier at dspnet.fr Mon Sep 24 16:48:11 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 15:48:11 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) Message-ID: <20120924134810.GA62723@kyoko.org> Hello, I don't know if it's been addressed before, but anyway : In my dovecot setup, I have local and virtual users. So, I need multiple passdb backends. Namely, passwd for the local users and ldap for the virtual users. passdb { driver = passwd } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } Everything work correctly : when a user logs in (imap/pop3) there's a lookup in passwd and if it fails there's a lookup in ldap (if I understand the process correctly), which eventually succeeds. Except that every time a virtual user logs in, dovecot logs an error, like : dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument I guess it's because the login is a full email that getpwnam fails. Anyway, the user logs in just fine. But I would like to know if/how I can get rid of the messages filling my logs ? I tried : auth_debug_passwords = no auth_verbose = no But no dice. I used dovecot 1.x before and there was no such messages. Thanks for any advice. K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From tss at iki.fi Mon Sep 24 17:16:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:16:06 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924134810.GA62723@kyoko.org> References: <20120924134810.GA62723@kyoko.org> Message-ID: On 24.9.2012, at 16.48, Philippe Chevalier wrote: > dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument > > I guess it's because the login is a full email that getpwnam fails. So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 386 bytes Desc: not available URL: From tss at iki.fi Mon Sep 24 17:32:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:32:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EED00.6090109@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: On 23.9.2012, at 14.05, Joe Auty wrote: > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? From blevi.linux at gmail.com Mon Sep 24 17:55:18 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 17:55:18 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <50607456.1040709@gmail.com> On 24/09/2012 17:32, Timo Sirainen wrote: > On 23.9.2012, at 14.05, Joe Auty wrote: > >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 > > Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Levi From tss at iki.fi Mon Sep 24 17:58:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:58:17 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <50607456.1040709@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> Message-ID: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> On 24.9.2012, at 17.55, Birta Levente wrote: > On 24/09/2012 17:32, Timo Sirainen wrote: >> On 23.9.2012, at 14.05, Joe Auty wrote: >> >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> (gdb) bt full >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> >> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >> > > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? From blevi.linux at gmail.com Mon Sep 24 18:04:29 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 18:04:29 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <5060767D.50002@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > I build my own rpm based on src rpm: dovecot-2.1.1-2_132.src.rpm. #dovecot -n auth_mechanisms = plain login cram-md5 debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no listen = * mail_access_groups = vmail mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = quota mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Quota exceeded, please contact postmaster at benvenuti.ro quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=85%% quota-warning 85 %u quota_warning4 = storage=80%% quota-warning 80 %u quota_warning5 = storage=50%% quota-warning 50 %u } postmaster_address = postmaster at mydomain.com service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0640 user = vmail } user = vmail } ssl_ca = References: <20120924134810.GA62723@kyoko.org> Message-ID: <20120924150440.GA85969@kyoko.org> On Mon, Sep 24, 2012 at 05:16:06PM +0300, Timo Sirainen wrote: >On 24.9.2012, at 16.48, Philippe Chevalier wrote: > >> dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument >> >> I guess it's because the login is a full email that getpwnam fails. > >So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? If I log in as a non-existent user (neither in passwd or ldap), without the domain part, it logs also an error, but this time from ldap : dovecot: auth: Error: ldap(foo.bar,xx.xx.xx.xx,): ldap_bind() failed: Invalid DN syntax My bind DN to check the password is : auth_bind_userdn = dc=%n,dc=%d,ou=Domains,ou=Mail,dc=dspnet,dc=fr (I have virtual users in multiple domains) So ldap protests probably because the "domain" part is missing. If I use a non-existent login "foo at bar", dovecot logs nothing : no error from passwd, no error from ldap, just an authentication error on the client side. I will apply the patch later today and will let you know the result. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From alessio at skye.it Mon Sep 24 18:40:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 17:40:52 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <50607F04.9040104@skye.it> Il 19/09/2012 15:07, Alessio Cecchi ha scritto: > Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>> >>>> LDA is configured and works fine but the problem is when the first >>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>> message is written to the user's Mailbox but the message remains, >>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>> the second attempt is delivered fine. >>> gdb backtrace would be very helpful in figuring out the problem: >>> http://dovecot.org/bugreport.html >>> >> Hi Timo, had you occasion to see the problem? Can I provide more information? Thanks > > This is the full bt: > > (gdb) bt full > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > ns = > ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = > 0x0, > v_modifiable = 0x0} > ids = 0x1928658 > i = > dest = > ret = -883075307 > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 > auser = 0x1949a08 > iter = 0x0 > acllist_path = 0x1928658 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" > ret = > ns = 0x1943e50 > output = 0x0 > st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = > 33152, > st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, > st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = > 1348059559, > tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, > st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, > 0, 0}} > path = 0x1928210 > > file_mode = 384 > dir_mode = 448 > gid = 4294967295 > list = > info = > rootdir = 0x1928610 "Sent" > origin = 0x194d178 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" > fd = 8 > #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) > at acl-backend-vfile-acllist.c:311 > acllist_path = > #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh > (backend=0x1944240) > at acl-backend-vfile-acllist.c:153 > __FUNCTION__ = "acl_backend_vfile_acllist_refresh" > #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, > name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 > acllist = > #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( > _aclobj=0x19444e0) at acl-backend-vfile.c:858 > old_validity = > validity = {global_validity = {last_check = 0, > last_read_time = 1348059559, last_mtime = 0, last_size = 0}, > local_validity = {last_check = 0, last_read_time = 0, > > last_mtime = 0, last_size = 0}, mailbox_validity = { > last_check = 0, last_read_time = 0, last_mtime = 0, > last_size = 0}} > mtime = 0 > ret = 26515976 > #6 0x00007f2fc9fc125e in acl_backend_get_default_rights > (backend=0x1944240, > mask_r=0x28) at acl-backend.c:164 > No locals. > #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at acl-mailbox-list.c:107 > alist = > nonowner_list_ctx = > ret = > backend = 0x1944240 > acl_mask = 0x1 > ns = 0x1943e50 > update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = > 0x7f2fcbf2ba88, > glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, > update_only = 0, match_parents = 0} > name = > #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, > flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 > _data_stack_cur_id = 2 > > ctx = 0x1946b20 > pool = > i = > inboxcase = > #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple > (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at mailbox-list-iter.c:158 > ctx = > ret = > __FUNCTION__ = "mailbox_list_iter_init_multiple" > #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, > pattern=, flags=1348059559) at > mailbox-list-iter.c:58 > patterns = {0x7f2fc9db76dc "*", 0x0} > #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, > bytes_r=, count_r=0x7fff362dfff0) at > quota-count.c:73 > ctx = 0x7f2fcb5beef3 > info = > #12 quota_count (root=0x1944950, bytes_r=, > count_r=0x7fff362dfff0) at quota-count.c:111 > i = 0 > ret = 0 > #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, > value_r=0x7fff362e0038) at quota-dict.c:113 > > dt = > bytes = 0 > count = 0 > #14 0x00007f2fc9db395a in dict_quota_update_callback ( > ret=, context=0x1949a08) at quota-dict.c:178 > value = 1 > #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, > line_r=) at dict-client.c:265 > ctx = 0x1984190 > #16 client_dict_read_one_line (dict=0x194c7f0, line_r= out>) > at dict-client.c:356 > id = 1 > line = > ret = 0 > __FUNCTION__ = "client_dict_read_one_line" > #17 0x00007f2fcb592565 in client_dict_wait (_dict=) > at dict-client.c:520 > dict = 0x194c7f0 > line = 0x0 > ret = > #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) > at quota-dict.c:90 > root = 0x1944950 > > #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 > pool = 0x194c5c0 > #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 > quota = 0x1944720 > i = 2 > #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) > at quota-storage.c:412 > quser = 0x1949a30 > quota_set = 0x1946f30 > #22 0x00007f2fcb88188e in mail_user_unref (_user=) > at mail-user.c:153 > user = 0x19483c0 > __FUNCTION__ = "mail_user_unref" > #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 > set_roots = {0x604640, 0x0} > ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, > dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, > src_envelope_sender = 0x0, dest_user = 0x0, > dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, > var_expand_table = 0x192e050, tried_default_save = true, > saved_mail = true, save_dest_mail = false, mailbox_full = > false, > > dsn = false} > service_flags = > user = 0x192d3c2 "cecchi10 at qboxdns.it" > errstr = 0x0 > path = 0x7fff362e0368 "\351\a" > storage_service = 0x192f3a0 > service_user = 0x192fd48 > service_input = {module = 0x4034d5 "lda", service = 0x4034d5 > "lda", > username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, > local_ip = {family = 0, u = {ip6 = {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = > {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, local_port = 0, remote_port = 0, > userdb_fields = 0x0, flags_override_add = 0, > flags_override_remove = 0, no_userdb_lookup = 0} > storage = 0x1943f30 > user_source = > destaddr_source = 0x403594 "user at hostname" > process_euid = > stderr_rejection = false > > ret = > c = > error = MAIL_ERROR_NONE > (gdb) > > Sorry > -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From feltrin at gmail.com Mon Sep 24 18:44:52 2012 From: feltrin at gmail.com (Jean Michel) Date: Mon, 24 Sep 2012 12:44:52 -0300 Subject: [Dovecot] Patches and dovecot releases Message-ID: I'd like to know if after a release, for example the recently release 2.1.10, its common to see few days from the release some bug reports and algo some patches, are theses patches applyed on the daily builds ? ------------------------------ Jean Michel Feltrin From m.naumann at globalview.de Mon Sep 24 19:32:43 2012 From: m.naumann at globalview.de (M. Naumann) Date: Mon, 24 Sep 2012 18:32:43 +0200 Subject: [Dovecot] Traffic Accounting Message-ID: <50608B2B.5050207@globalview.de> Hi, I'm trying to find out how to do traffic accounting with Dovecot 2.x, preferrably v2.0.9, preferrably on CentOS 6. I've previously asked on IRC, but there was little feedback, and my understanding is now this list is the preferred media for such inquiries. If I recall correctly, some weeks ago I was told that traffic accounting is not officially supported on Dovecot 2, but that there could still be ways to get it to work, but no details were provided. I can think of the following approaches: * rawlog, preferrably piped (if that's possible?) into something like wc to prevent privacy issues and to reduce the I/O overhead * maildrop filtering in front of dovecot LDA (for mail inbound to mail storage) * sieve filtering Unfortunately I have little experience with either so far, so it's hard to make a good choice. I would appreciate hints on these approaches, and on any other approaches you can think of, as well as any related documentation / how-to you could point me to. While I'm subscribed to the list (for mail authentication purposes), I've disabled receiving any e-mail form the list, so please CC me on any replies. Thanks in advance, Moritz From robert at schetterer.org Mon Sep 24 19:46:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:46:23 +0200 Subject: [Dovecot] Patches and dovecot releases In-Reply-To: References: Message-ID: <50608E5F.2080704@schetterer.org> Am 24.09.2012 17:44, schrieb Jean Michel: > I'd like to know if after a release, for example the recently release > 2.1.10, its common to see few days from the release some bug reports and > algo some patches, are theses patches applyed on the daily builds ? > > > ------------------------------ > Jean Michel Feltrin > look http://hg.dovecot.org/ dovecot has tons of config options, and there are extrem multi ways to set it up ( for this we love dovecot ), so some stuff may bug in a stable release for some setups/functions sometimes also distros do self patching, there are sometimes kernel bugs etc in general its not a software having bugs, its the way and speed they get fixed, and Timo does a "wonderman" job here after all ,unless there are no security fixes, or you urgent need new setup options etc, nobody presses you to upgrade in a stable tree read the changelogs -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Sep 24 19:48:38 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:48:38 +0200 Subject: [Dovecot] Traffic Accounting In-Reply-To: <50608B2B.5050207@globalview.de> References: <50608B2B.5050207@globalview.de> Message-ID: <50608EE6.6030905@schetterer.org> Am 24.09.2012 18:32, schrieb M. Naumann: > Hi, > > I'm trying to find out how to do traffic accounting with Dovecot 2.x, > preferrably v2.0.9, preferrably on CentOS 6. > > I've previously asked on IRC, but there was little feedback, and my > understanding is now this list is the preferred media for such > inquiries. If I recall correctly, some weeks ago I was told that traffic > accounting is not officially supported on Dovecot 2, but that there > could still be ways to get it to work, but no details were provided. > > I can think of the following approaches: > > * rawlog, preferrably piped (if that's possible?) into something like wc > to prevent privacy issues and to reduce the I/O overhead > > * maildrop filtering in front of dovecot LDA (for mail inbound to mail > storage) > > * sieve filtering > > Unfortunately I have little experience with either so far, so it's hard > to make a good choice. I would appreciate hints on these approaches, and > on any other approaches you can think of, as well as any related > documentation / how-to you could point me to. > > While I'm subscribed to the list (for mail authentication purposes), > I've disabled receiving any e-mail form the list, so please CC me on any > replies. > > Thanks in advance, > > Moritz > perhaps this helps on the 2.1.x tree there is a stats plugin http://wiki2.dovecot.org/Statistics and/or use syslog analysers etc -- Best Regards MfG Robert Schetterer From ben at morrow.me.uk Mon Sep 24 20:01:02 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:01:02 +0100 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> Message-ID: <20120924170101.GB63656@anubis.morrow.me.uk> At 2PM +0300 on 24/09/12 you (Timo Sirainen) wrote: > On 24.9.2012, at 14.44, A.L.E.C wrote: > > > next is "run" or "pipe", but what if you create global separator option > > and detect multi-command syntax usage automatically without a keyword? > > > > Syntax for doveadm would be > > > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > > [command_options] [command_arguments] [separator command > > [command_options] [command_arguments] [...]] > > > > and example > > > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge > > Hmm. Yes, that might work. Although it would have to be: > > doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge > > because both -A and -s are mail command specific parameters, which > won't work for non-mail commands. > > Hmm. This reminds me also that it would be possible with some extra > work to do some command interaction. IMAP supports saving search > results, which can later be accessed with $ parameter. So this could > be made to work: > > doveadm search -s : from foo : fetch text \$ : expunge \$ This is turning into a proper scripting language, so perhaps something like doveadm -e 'search from foo; fetch text $; expunge $' with 'doveadm -F file' to run a script file? Ben From ben at morrow.me.uk Mon Sep 24 20:11:23 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:11:23 +0100 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505FFAE7.3070905@um.es> References: <505C3435.30601@blueboard.cz> <505FFAE7.3070905@um.es> Message-ID: <20120924171123.GC63656@anubis.morrow.me.uk> > El 21/09/12 11:32, Tom?? Randa escribi?: > > Hello, > > > > I still cannot get dovecot running with more then 1000 processes, but > > hard limit is 8192 per user in box. I tried everything, including > > modifying startup script of dovecot to set ulimit -u 8192. Could it be > > some dovecot bug or dovecot<>freebsd bug? > > I also tried to set client_limit=2 in imap service to spawn more imap > > clients in one process, but still I am over 1000 processes with kernel > > message: > > > > maxproc limit exceeded by uid 89 You may be running into the kern.maxprocperuid sysctl setting. This is initialized to 9/10ths of kern.maxproc, but can be changed independantly. If you do this you may want to consider setting a default maxproc rlimit in login.conf for the other users on the box. (You may, of course, already have a maxproc limit in login.conf, and that's what's causing the problem, though the default install doesn't include one.) If you have procfs mounted you can check the maxproc rlimit of a running process by looking in /proc/pid/rlimit. In principle it's possible to also get this information with libkvm, but it's not very easy and I don't think any of the standard utilities expose it. Ben From alessio at skye.it Mon Sep 24 20:19:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 19:19:30 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <50607F04.9040104@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <50607F04.9040104@skye.it> Message-ID: <50609622.7060702@skye.it> Il 24/09/2012 17:40, Alessio Cecchi ha scritto: > Il 19/09/2012 15:07, Alessio Cecchi ha scritto: >> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >>> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>>> >>>>> LDA is configured and works fine but the problem is when the first >>>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>>> message is written to the user's Mailbox but the message remains, >>>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>>> the second attempt is delivered fine. >>>> gdb backtrace would be very helpful in figuring out the problem: >>>> http://dovecot.org/bugreport.html >>>> >>> > > Hi Timo, > > had you occasion to see the problem? Can I provide more information? > > Thanks After further testing I found this behavior, a note, I'm using dict quota in mysql. - add a new user - delivery the first email via deliver - "Segmentation fault" - I remove the newly created user - add the same user - delivery the first email via deliver - OK - add a new user - the user connects via pop/imap - delivery the first email via deliver - OK - add a new user - manually create the entry for dict quota <<== - delivery the first email via deliver - OK it seems that if there is user's entry in the dict database the problem does not appear. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stsiol at yahoo.co.uk Mon Sep 24 20:42:35 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 18:42:35 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb Message-ID: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, Xmail and OpenLDAP for a company and serving about 40 acouunts. The machine is wonderful. I am very happy with it. However, I am running out of disk space. It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity has reached 82%.? I am starting of getting nervous. Does anyone know of a painless way to migrate the entire contents directly to another pair of 146Gb SCSI RAID1 disks ? I thought of downtime and using clonezilla, but my last experience with it was questionable. I remember having problems declaring disk re-sizing from the smaller capacity drives to the larger ones. CentOS 5.5 Manual install of : Mysql XMail (pop3/smtp) ASSP (anti spam) Apache / LAMP and last but by no means list : Dovecot Dovecot -n : # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 96 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd Any help would be appreciated or any ideas you might have. Regards, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From andrei.michescu at miau.ca Mon Sep 24 21:01:33 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:01:33 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Hello Spyros, As "best practice" you never have the OS and the data/logs/user homes on the same partition or set of disks. If this is the case then your life is pretty easy: -simply create the new set of partitions -mount the new ones in a temporary location -rsync (or copy everything from old partitions) -Stop dovecot / all other daemons that might be using the data -mount the new ones in the place of old ones, mount the old ones in the place of new ones - rsync again (should be quick as not many things changed) - start all your deamons again :P If you do not have separate partitions maybe this is the perfect time to look into that... I would also look into btrfs... might be a good pick for your new partitions. best regards, Andrei > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%.? > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > ? verbose: yes > ? debug: yes > ? debug_passwords: yes > ? passdb: > ? ? driver: passwd-file > ? ? args: /etc/dovecot/passwd > ? passdb: > ? ? driver: pam > ? userdb: > ? ? driver: static > ? ? args: uid=vmail gid=vmail home=/home/vmail/%u > ? userdb: > ? ? driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis? > > !DSPAM:50609d2c301831828332458! > > From robert at schetterer.org Mon Sep 24 21:06:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:06:18 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A11A.9000007@schetterer.org> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > rsync should do the job depending on your whole machine setup it might only be only umount old /home and mount new(bigger) /home after sync ,perhaps with tmp store elsewhere ( for sure you have to have a plan before doing..) but your dovecot is very outdated, i would recommend get up to new hard and software/os install, and then migrate to new machine > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From lists at wildgooses.com Mon Sep 24 21:07:00 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:07:00 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A144.9040607@wildgooses.com> This is one of those questions which is almost too easy if you are familiar with Linux. Trying not to sound like a d*ck, but is it an option to rent someone to help with admin jobs? For example, were it me then I would probably have setup some partitioning scheme with separate partitions for data and operating system? Possibly also using LVM? You have several options, mainly the choice of filesystem will dictate here, but quite possibly you can: 1) Pull the drives one by one and rebuild the raid after each. Keep the old drives since you can technically roll back onto them. Expand the partitions (scary without LVM) and then expand the filesystem on the partitions 2) Boot from a DVD/Flash on your favourite rescue distro (I like sysrecuecd). Create the new raid, copy the old to the new, remove the old drives, reboot from new. Possibly taking the time to repartition and move some data around while you do it (remember to update fstab) Both are fairly simple if you have done it once, but it would be well worth finding someone either local or who will log in via remote control and support you? Final thought: For the size of drives you are looking at, SSD drives are relatively inexpensive and likely comparable with the high end drives you are probably looking to buy? For 40 users I would hazard a guess you likely would be happy with inexpensive low end drives, but certainly a couple of small SSDs will blow away a spinning disk and give you a decent upgrade... Good luck Ed W On 24/09/2012 18:42, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From andrei.michescu at miau.ca Mon Sep 24 21:07:11 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:07:11 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Message-ID: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Hello Spyros, Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA connector... so the solution below does not really apply to you :( Andrei > Hello Spyros, > > As "best practice" you never have the OS and the data/logs/user homes on > the same partition or set of disks. > > If this is the case then your life is pretty easy: > -simply create the new set of partitions > -mount the new ones in a temporary location > -rsync (or copy everything from old partitions) > -Stop dovecot / all other daemons that might be using the data > -mount the new ones in the place of old ones, mount the old ones in the > place of new ones > - rsync again (should be quick as not many things changed) > - start all your deamons again :P > > If you do not have separate partitions maybe this is the perfect time to > look into that... > > I would also look into btrfs... might be a good pick for your new > partitions. > > best regards, > Andrei > >> Hello all, >> >> I have a DL360 G4 1U server that does a wonderfull job with dovecot >> horde, >> Xmail and OpenLDAP for a company and serving about 40 acouunts. >> >> The machine is wonderful. I am very happy with it. >> However, I am running out of disk space. >> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >> has reached 82%.? >> >> I am starting of getting nervous. >> >> Does anyone know of a painless way to migrate the entire contents >> directly >> to another pair of 146Gb SCSI RAID1 disks ? >> >> I thought of downtime and using clonezilla, but my last experience with >> it >> was questionable. I remember having problems declaring disk re-sizing >> from the smaller capacity drives to the larger ones. >> >> CentOS 5.5 >> Manual install of : >> >> Mysql >> XMail (pop3/smtp) >> ASSP (anti spam) >> Apache / LAMP >> and last but by no means list : Dovecot >> >> Dovecot -n : >> >> # 1.2.16: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >> base_dir: /var/run/dovecot/ >> log_path: /var/log/dovecot/dovecot.log >> info_log_path: /var/log/dovecot/dovecot-info.log >> ssl_parameters_regenerate: 48 >> verbose_ssl: yes >> login_dir: /var/run/dovecot//login >> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >> login_greeting: * Dovecot ready * >> login_max_processes_count: 96 >> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >> mail_plugins: zlib >> auth default: >> ? verbose: yes >> ? debug: yes >> ? debug_passwords: yes >> ? passdb: >> ? ? driver: passwd-file >> ? ? args: /etc/dovecot/passwd >> ? passdb: >> ? ? driver: pam >> ? userdb: >> ? ? driver: static >> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >> ? userdb: >> ? ? driver: passwd >> >> >> Any help would be appreciated or any ideas you might have. >> >> Regards, >> >> spyros >> >> >> >> >> >> >> ---- >> "I merely function as a channel that filters >> music through the chaos of noise" >> - Vangelis? >> >> >> >> > > > > !DSPAM:5060a006309197419291868! > > From robert at schetterer.org Mon Sep 24 21:23:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:23:25 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <5060A51D.80404@schetterer.org> Am 24.09.2012 20:07, schrieb Michescu Andrei: > Hello Spyros, > > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( > > Andrei depends how long downtime is acceptable i.e go rsync tmp over usb storage, or simple tmp nfs mounts to other servers are thinkable to minimize downtime should be no big problem its only one server with maildir and less mailboxes and data but thats all much off topic with dovecot -- Best Regards MfG Robert Schetterer From chevalier at dspnet.fr Mon Sep 24 21:41:57 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 20:41:57 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924150440.GA85969@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> Message-ID: <20120924184157.GA75341@kyoko.org> On Mon, Sep 24, 2012 at 05:04:40PM +0200, Philippe Chevalier wrote: > >I will apply the patch later today and will let you know the result. I applied the patch, and obviously, when getpwnam_r sets the result to NULL and returns EINVAL, dovecot do as if the entry was not found and stays mute. So, thank you, auth is now a lot less noisy. As for the ldap message, it errors if there's no domain in the login. In the doc, it says that %d is empty if there's no domain part. So I guess it's an enhancement request : a configuration option to have it filled out with a default domain if there's no one supplied by the client. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From stsiol at yahoo.co.uk Mon Sep 24 21:42:21 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 19:42:21 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> ----- Original Message ----- > From: Michescu Andrei > To: Dovecot Mailing List > Cc:? > Sent: Monday, 24 September 2012, 21:07 > Subject: Re: [Dovecot] 76Gb to 146Gb >? > Hello Spyros, >? > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( >? > Andrei >? >> Hello Spyros, >>? >> As "best practice" you never have the OS and the data/logs/user? > homes on >> the same partition or set of disks. >>? >> If this is the case then your life is pretty easy: >> ? -simply create the new set of partitions >> ? -mount the new ones in a temporary location >> ? -rsync (or copy everything from old partitions) >> ? -Stop dovecot / all other daemons that might be using the data >> ? -mount the new ones in the place of old ones, mount the old ones in the >> place of new ones >> ? - rsync again (should be quick as not many things changed) >> ? - start all your deamons again :P >>? >> If you do not have separate partitions maybe this is the perfect time to >> look into that... >>? >> I would also look into btrfs... might be a good pick for your new >> partitions. >>? >> best regards, >> Andrei >>? >>> Hello all, >>>? >>> I have a DL360 G4 1U server that does a wonderfull job with dovecot >>> horde, >>> Xmail and OpenLDAP for a company and serving about 40 acouunts. >>>? >>> The machine is wonderful. I am very happy with it. >>> However, I am running out of disk space. >>> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >>> has reached 82%.? >>>? >>> I am starting of getting nervous. >>>? >>> Does anyone know of a painless way to migrate the entire contents >>> directly >>> to another pair of 146Gb SCSI RAID1 disks ? >>>? >>> I thought of downtime and using clonezilla, but my last experience with >>> it >>> was questionable. I remember having problems declaring disk re-sizing >>> from the smaller capacity drives to the larger ones. >>>? >>> CentOS 5.5 >>> Manual install of : >>>? >>> Mysql >>> XMail (pop3/smtp) >>> ASSP (anti spam) >>> Apache / LAMP >>> and last but by no means list : Dovecot >>>? >>> Dovecot -n : >>>? >>> # 1.2.16: /etc/dovecot/dovecot.conf >>> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >>> base_dir: /var/run/dovecot/ >>> log_path: /var/log/dovecot/dovecot.log >>> info_log_path: /var/log/dovecot/dovecot-info.log >>> ssl_parameters_regenerate: 48 >>> verbose_ssl: yes >>> login_dir: /var/run/dovecot//login >>> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >>> login_greeting: * Dovecot ready * >>> login_max_processes_count: 96 >>> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >>> mail_plugins: zlib >>> auth default: >>> ? verbose: yes >>> ? debug: yes >>> ? debug_passwords: yes >>> ? passdb: >>> ? ? driver: passwd-file >>> ? ? args: /etc/dovecot/passwd >>> ? passdb: >>> ? ? driver: pam >>> ? userdb: >>> ? ? driver: static >>> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >>> ? userdb: >>> ? ? driver: passwd >>>? >>>? >>> Any help would be appreciated or any ideas you might have. >>>? >>> Regards, >>>? >>> spyros >>>? >>>? >>>? >>>? >>>? >>>? >>> ---- >>> "I merely function as a channel that filters >>> music through the chaos of noise" >>> - Vangelis? >>>? >>>? >>>? >>>? >>? >>? >>? >> !DSPAM:5060a006309197419291868! Andrei, Thank you very much for you kind reply and both your messages. Having said that, would it be possible to take away on 72Gb drive (say Drive1 the second drive) and shove in one of the two 146Gb ones ? Shouldn't the array be rebuilt ? Will it use the extra disk space though ? Thanks, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From lists at wildgooses.com Mon Sep 24 21:55:07 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:55:07 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A144.9040607@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> Message-ID: <5060AC8B.7090703@wildgooses.com> On 24/09/2012 19:07, Ed W wrote: > This is one of those questions which is almost too easy if you are > familiar with Linux. Trying not to sound like a d*ck, but is it an > option to rent someone to help with admin jobs? For example, were it > me then I would probably have setup some partitioning scheme with > separate partitions for data and operating system? Possibly also using > LVM? That came out wrong... What I meant to say was something more like "if you were to employ someone locally they would probably give you a whole bunch of ideas on how you could adjust the setup of the server to be more future proof. It would be worth working with someone just to get that right. For example, here are some ideas that occur to me that you could use ..." Sorry, should re-read my words before hitting send Ed From joe at netmusician.org Mon Sep 24 22:03:46 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 24 Sep 2012 15:03:46 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <5060AE92.5040904@netmusician.org> > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? Thanks Timo, I have done so. Here is the results of my debugging info now: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap-login...done. (no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 hash_table_destroy (_table=0x28) at hash.c:106 106 hash.c: No such file or directory. in hash.c (gdb) bt full #0 hash_table_destroy (_table=0x28) at hash.c:106 table = #1 0x00007ff300721054 in settings_parser_deinit (_ctx=) at settings-parser.c:237 ctx = 0x0 #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) at master-service-settings-cache.c:86 cache = 0x9f9a60 entry = 0xa016e0 next = 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x00007ff3009a5018 in main_deinit (binary=, argc=2, argv=0x9f8370) at main.c:355 No locals. #4 login_binary_run (binary=, argc=2, argv=0x9f8370) at main.c:407 set_pool = 0x9f8a30 allow_core_dumps = login_socket = c = #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #6 0x0000000000402459 in _start () No symbol table info available. (gdb) > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From stsiol at yahoo.co.uk Mon Sep 24 22:13:50 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:13:50 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060AC8B.7090703@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> <5060AC8B.7090703@wildgooses.com> Message-ID: <1348514030.72104.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: Ed W > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:55 > Subject: Re: [Dovecot] 76Gb to 146Gb > > On 24/09/2012 19:07, Ed W wrote: >> This is one of those questions which is almost too easy if you are familiar > with Linux.? Trying not to sound like a d*ck, but is it an option to rent > someone to help with admin jobs?? For example, were it me then I would probably > have setup some partitioning scheme with separate partitions for data and > operating system? Possibly also using LVM? > > That came out wrong...? What I meant to say was something more like "if you > were to employ someone locally they would probably give you a whole bunch of > ideas on how you could adjust the setup of the server to be more future proof.? > It would be worth working with someone just to get that right.? For example, > here are some ideas that occur to me that you could use ..." > > Sorry, should re-read my words before hitting send > > Ed > Ed, Don't worry about it. I wasn't offended. I have a lot of experience with linux but not on heavy metal servers. I used to have plenty of experience back in the G2/ G3 era (I was also ACE in the Compaq years) but that was back in the time that Compaq was only supporting Windows OSs and SCO. Also the problem is that I don't have the time to "play" with a spare HP/Compaq server ( I have a couple laying around btw). I'll get round to it at some point. I am just asking you chaps because I am sure people out there had the chance to tinker with newer and better equipment. Thank you for your reply, Best Regards, spyros ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Sep 24 22:24:37 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:24:37 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A11A.9000007@schetterer.org> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> Message-ID: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> ----- Original Message ----- > From: Robert Schetterer > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:06 > Subject: Re: [Dovecot] 76Gb to 146Gb > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >> Hello all, >> > ----%<----%<----%<----%<----%<----%<----%<----%<---- >? > rsync > should do the job > > depending on your whole machine setup it might only be only > umount old /home and mount new(bigger) /home after sync > ,perhaps with tmp store elsewhere > ( for sure you have to have a plan before doing..) > > but your dovecot is very outdated, i would recommend > get up to new hard and software/os install, and then migrate > to new machine > >> > > > -- > Best Regards > MfG Robert Schetterer > On client machines I have thunderbird. What if : 1. I would make sure that thunderbird keeps a local copy of all the message (I think there is a check box somewhere on settings) 2. Make sure all client machines have synced their mailboxes locally on thunderbird. 3. Install a new version of Dovecot/Horde/XMail etc. 4. When the new installation is done, try to sync from the existing clients pc's to dovecot ? Would that work ? It's one scenario I am seriously contemplating. Thank you very much again, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From me at junc.org Mon Sep 24 23:59:00 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 22:59:00 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <6ebfff9b6a3d0faeb0a91719e638e232@junc.org> Spyros Tsiolis skrev den 24-09-2012 19:42: > Any help would be appreciated or any ideas you might have. try google "centos cloud server" if you would like to do it local, use all 4 drives with 2 raid1 in the same controller if possible, then use sysrescue cd to tar it all over to the other raid1 while its down no matter how, it will be downtime i am not using centos here so i cant be more specifik http://www.sysresccd.org From me at junc.org Tue Sep 25 00:09:31 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 23:09:31 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <828720e5142c1cb59e96bcd9a15b0efb@junc.org> Spyros Tsiolis skrev den 24-09-2012 20:42: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? this can be done yes, but you will have to do more steps :) first step, remove one drive add the 146 drive wait it for rebuildin when done, remove the last small drive add the last 146 drive wait for it to rebuild now at this stage you have 72g more unused room for new partions make this new partion /home and after its being created, move the user data to it, but this leves 72g system partion with just few gigs needed ?, then i would create the new partion as lwm2, and then possible shrink system, and mount the lwm2 as /home, that way you have more options later if 146 will be to small again warn i have not doing this myself, but if should work in teori atleast From daniel.parthey at informatik.tu-chemnitz.de Tue Sep 25 00:24:48 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 24 Sep 2012 23:24:48 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: <20120924212448.GA9310@daniel.localdomain> Timo Sirainen wrote: > On 21.9.2012, at 8.28, Timo Sirainen wrote: > > >> Timo Sirainen wrote: > >>> doveadm multi [-A | -u wildcards] [ [...]] > >>> > >>> Thoughts? > > > >> As command name I could also think of "doveadm sequence", which > >> implies the commands being executed in serial order. > > > > Hmm. Maybe. > > "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. Ok, so how about "batch"? It reads a series of commands and collects them into one "batch job" which is then carried out. http://en.wikipedia.org/wiki/Batch_(Unix) Regards Daniel -- https://plus.google.com/103021802792276734820 From stan at hardwarefreak.com Tue Sep 25 01:05:35 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 24 Sep 2012 17:05:35 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <5060D92F.2070106@hardwarefreak.com> On 9/24/2012 1:42 PM, Spyros Tsiolis wrote: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? It's always best to manually take a drive off line before pulling it. > Shouldn't the array be rebuilt ? Depends on how your 6i is configured. Best guess is that it will automatically rebuild the mirror on the new 146GB drive, but... > Will it use the extra disk space though ? It will probably not. You need to read the 6i manual. I sense a hardware upgrade in your near future, either an HP server with 4 bays, or an SFF8088 JBOD chassis and an inexpensive RAID card. You already have the 146GB drives correct? They are HP pluggable drives? Which means they only work in HP gear. If that's the case you need a new server with at least 4 drive bays. You you need to buy an off brand JBOD box and two standard SATA drives. Or maybe your organizations needs more storage on many servers, and it's time to step up to an iSCSI SAN array. -- Stan From michael at orlitzky.com Tue Sep 25 05:59:15 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 22:59:15 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50611E03.3040706@orlitzky.com> On 09/24/2012 01:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. We've done this on the same hardware. You can pick up these servers for cheap; just buy an extra one. Take the new machine, throw two big disks in it, and install Gentoo. Rsync the important stuff. Make sure all of the services are working on the new machine. When you're ready to make the switch, disable external networking on the current live server. Rsync everything again, and then turn the old server off. Add its IP address to the new server. Maybe kick your router's ARP cache to expedite the change. It should only cause a minute or two of downtime. From michael at orlitzky.com Tue Sep 25 06:02:55 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 23:02:55 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <50611E03.3040706@orlitzky.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <50611E03.3040706@orlitzky.com> Message-ID: <50611EDF.2060602@orlitzky.com> On 09/24/2012 10:59 PM, Michael Orlitzky wrote: > > We've done this on the same hardware. You can pick up these servers for > cheap; just buy an extra one. Take the new machine, throw two big disks > in it, and install Gentoo. I seem to have gone insane, I thought this was on gentoo-user for some reason. Anyway, it's a fine suggestion =) From robert at schetterer.org Tue Sep 25 08:41:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 25 Sep 2012 07:41:26 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <50614406.50400@schetterer.org> Am 24.09.2012 21:24, schrieb Spyros Tsiolis: > > > > > ----- Original Message ----- >> From: Robert Schetterer >> To: dovecot at dovecot.org >> Cc: >> Sent: Monday, 24 September 2012, 21:06 >> Subject: Re: [Dovecot] 76Gb to 146Gb >> >> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >>> Hello all, >>> >> ----%<----%<----%<----%<----%<----%<----%<----%<---- >> >> rsync >> should do the job >> >> depending on your whole machine setup it might only be only >> umount old /home and mount new(bigger) /home after sync >> ,perhaps with tmp store elsewhere >> ( for sure you have to have a plan before doing..) >> >> but your dovecot is very outdated, i would recommend >> get up to new hard and software/os install, and then migrate >> to new machine >> >>> >> >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > On client machines I have thunderbird. > > What if : > > 1. I would make sure that thunderbird keeps a local > copy of all the message (I think there is a check box > somewhere on settings) > > 2. Make sure all client machines have synced their > mailboxes locally on thunderbird. > > > 3. Install a new version of Dovecot/Horde/XMail etc. > > 4. When the new installation is done, try to sync > from the existing clients pc's to dovecot ? > > Would that work ? > It's one scenario I am seriously contemplating. > > Thank you very much again, > > s. > in short words ,dont do it like this setup your new server, test it, then do i.e imapsync from old to new switch your ips then ,done only one way which you can go look archives and www for migration tips > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From joh.hendriks at gmail.com Tue Sep 25 10:34:02 2012 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Tue, 25 Sep 2012 09:34:02 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50615E6A.5030109@gmail.com> Spyros Tsiolis schreef: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis A solution you could try is add a new server to the mix. Leave the OS on the DL360 and put the mail data on a second machine. Then use NFS to mount the new data store. If you use a ZFS solution, expanding your data store is a no brainer. 146 GB drives are expensive, for 40 users SATA drives can do the job without any problem. gr Johan Hendriks From voytek at sbt.net.au Tue Sep 25 11:07:23 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Tue, 25 Sep 2012 18:07:23 +1000 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? Message-ID: I have a user who occasionally reads, then deletes emails using his phone, on his and others email accounts, causing emails deleted on mail server, then he says 'oops, can you recover emails I've deleted?' what's the best way to duplicate all emails to duplicate accounts ? backup folders ? so, next time he deletes emails they can be 'recovered' on the server ? I was thinking along the lines of duplicating ALL emails...? any tips and pointers appreciated -- Voytek From Ralf.Hildebrandt at charite.de Tue Sep 25 11:23:53 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 25 Sep 2012 10:23:53 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <20120925082353.GF7796@charite.de> * Voytek Eymont : > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? > > I was thinking along the lines of duplicating ALL emails...? > > any tips and pointers appreciated http://wiki.dovecot.org/Plugins/Lazyexpunge -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From h.reindl at thelounge.net Tue Sep 25 11:24:18 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 25 Sep 2012 10:24:18 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <50616A32.7040808@thelounge.net> Am 25.09.2012 10:07, schrieb Voytek Eymont: > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? configure his phone to leave messages on the server if this is the case and he is deleting them manually explain him that it is NOT your problem if he is too stupid to use email at all and sooner or later he will learn it -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From marc at perkel.com Wed Sep 26 00:34:55 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 25 Sep 2012 14:34:55 -0700 Subject: [Dovecot] Dovecot Authentication through different IMAP server? Message-ID: <5062237F.4000000@perkel.com> Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? Here's what I'm thinking. I run a spam filtering service and thinking about a way to give users access to their stored spam on my servers. However real email is forwarded to their server, So I'm wondering if I can configure dovecot somehow to do a back end authentication to their email account to gain access to their spam. Open to suggestions From noel.butler at ausics.net Wed Sep 26 04:23:11 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 26 Sep 2012 11:23:11 +1000 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1348622591.6847.23.camel@tardis> Spyros, Sounds like you have bigger problems, as you appear to have no disaster recovery processes, since your using a standalone server, as recommended to you earlier by Robert, rsync is your friend, and not just now, but daily :) setup a rolling 7 day rsync archive (sata disks are big, cheap and good enough for backups), and dont just rsync the same stuff to the same place every night, thats no good if an error occurs and wipes out your mail, coz rsync will only see it gone, and clean it out of your backup. So long as RedHat have patched dovecot for bugs and still does, 1.2.16 would still be fine, dovecot-1.2.17 is latest 1.2.x series but IIRC no longer supported, but thats fine, there are a lot, and I mean, a LOT of companies and ISP's running that version, moving to 2.x was never going to happen with new releases coming every week or so at one stage, and as 2.1.x doesn't now have nearly as regular updates, I recently myself consider 2.1 finally to be mature enough to put on production servers, which I will likely do around Christmas when its quiet. Lastly, never ever ever rely on your users to have a copy of their mail as a recovery method, it wont work, some will fsck it up, some wont bother, and you will be blamed for not having redundancy inplace. On Mon, 2012-09-24 at 20:24 +0100, Spyros Tsiolis wrote: > > > > ----- Original Message ----- > > From: Robert Schetterer > > To: dovecot at dovecot.org > > Cc: > > Sent: Monday, 24 September 2012, 21:06 > > Subject: Re: [Dovecot] 76Gb to 146Gb > > > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > >> Hello all, > >> > > ----%<----%<----%<----%<----%<----%<----%<----%<---- > > > > rsync > > should do the job > > > > depending on your whole machine setup it might only be only > > umount old /home and mount new(bigger) /home after sync > > ,perhaps with tmp store elsewhere > > ( for sure you have to have a plan before doing..) > > > > but your dovecot is very outdated, i would recommend > > get up to new hard and software/os install, and then migrate > > to new machine > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From stan at hardwarefreak.com Wed Sep 26 09:34:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 26 Sep 2012 01:34:55 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> <5060D92F.2070106@hardwarefreak.com> <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <5062A20F.8060500@hardwarefreak.com> On 9/25/2012 12:29 AM, Spyros Tsiolis wrote: > My clients don't invest in hardware. I live in Greece. Things are really > bad right now. Not that if they were better they (the client) would invest > in their infrastructure. ... > Thank you very much Stan for you kind reply. > > I reply to you in private since I was told to stop the thread > on the dovecot list because it's off-topic. I'm going to bring it back on topic. To recap the issue: 1. 73GB volume, ~82% full. Volume contains OS, spool, and mailboxes 2. Two 146GB drives on hand that are impossible to employ in either a live or offline migration due to hardware limitations--drive slots (0.82*73GB=60GB)-5GB(OS)= ~55GB/40= ~1.375GB/user mailbox (This math assumes the box has already been scoured and things like source files and other temporary stuff has already been removed, along with anything and everything not necessary to the primary functions of this box) 1.375GB/user suggests these users seem to horde old emails, or simply don't keep their mailboxes tidy. Normally I'd guess that a lot of space in a situation like this is taken up by full text search indexes, but you don't appear to have these enabled. I notice in your 'dovecot -n' output that you are already employing zlib, but to what degree is currently unknown to the list. Are you compressing maildirs with a nightly script? If not, and you're compressing on an ad hoc basis, realtime compression during write could save you quite a bit of additional space, extending the operating life of your current drives. Of course a simple request to your users to clean up their mailboxes would surely help as well, if that's a possibility--some users would flog one for making such a suggestion. If you're not fully taking advantage of compression currently, you may want to consider upgrading to Dovecot 2.x which allows transparent compression to dbox and maildir storage. You may want to migrate to mdbox message store format while still using zlib, and enable single instance attachment storage (SIS). As the name suggests, SIS saves an attachment only once, regardless how many user emails in which it exists. See: http://wiki2.dovecot.org/Plugins/Zlib I'd point you to the docs for SIS but I'm unable to find them in the Wiki. Surely someone else can provide the settings and/or update the wiki. -- Stan From robert at schetterer.org Wed Sep 26 09:39:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 08:39:27 +0200 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <5062A31F.9070705@schetterer.org> Am 25.09.2012 23:34, schrieb Marc Perkel: > Just wondering - is there any way for a dovecot server to authenticate > against a different IMAP server? > > Here's what I'm thinking. I run a spam filtering service and thinking > about a way to give users access to their stored spam on my servers. > However real email is forwarded to their server, So I'm wondering if I > can configure dovecot somehow to do a back end authentication to their > email account to gain access to their spam. > > Open to suggestions > i am not sure your idea is the best way to goal your target, running a i.e amavis filter gateway before postfix with some style of quarantaine seems to me a better idea however sasl can do auth against imap i.e ( attention examples may be outdated or different by other distro config style and/or dovecot versions) MECH=?rimap? FLAGS=?-O imap.server.name? http://blog.travisz.com/2010/12/17/debian-mail-server-with-postfix-dovecot-and-saslauthd/ #vim /etc/dovecot/dovecot.conf Look for a line that starts with ?auth default?. Once you fine that line, insert the following: auth default { mechanisms = plain login passdb pam { } userdb passwd { } socket listen { client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix there might better possibilities, wait until more response from the list -- Best Regards MfG Robert Schetterer From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 13:50:44 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 12:50:44 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder Message-ID: <5062DE04.3020601@TU-Cottbus.De> Hi Timo, hi all, we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had configured it in a test environemnt like described in "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" configuration. All actions on first level directories ".INBOX", ."Sent" or ".Archives" are working as expected. If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an error message in dovecot log: Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open expunge mailbox: Character not allowed in mailbox name: '.' What can we do or change to make it work? -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From amateo at um.es Wed Sep 26 13:55:56 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 26 Sep 2012 12:55:56 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server Message-ID: <5062DF3C.3050601@um.es> Hello, I have a infrastructure using director in front of backend servers. So I normally run doveadm commands in director servers, not the backends. The problem I have is when I try to concatenate commands like this: doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; done The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: e62e0d3834ed094e5c7900007efb8a67 66 e62e0d3834ed094e5c7900007efb8a67 71 e62e0d3834ed094e5c7900007efb8a67 74 e62e0d3834ed094e5c7900007efb8a67 75 e62e0d3834ed094e5c7900007efb8a67 77 e62e0d3834ed094e5c7900007efb8a67 78 e62e0d3834ed094e5c7900007efb8a67 so I can't postprocess correctly the output. I think it could be a bug in the communication between the doveadm client and server. PS: As the time of writing this email, I have realized that I could use fetch command directly. But, anyway, I want to report the problem/bug so it can be fixed. From maciej.uhlig at us.edu.pl Wed Sep 26 14:52:18 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 13:52:18 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line Message-ID: <5062EC72.5040505@us.edu.pl> This is a dovecot sieve log line: 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: msgid=: stored mail into mailbox 'INBOX' Would it be possible to have mail size logged here? MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From robert at schetterer.org Wed Sep 26 15:51:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:51:00 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062DE04.3020601@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> Message-ID: <5062FA34.7020502@schetterer.org> Am 26.09.2012 12:50, schrieb Andreas Kossack: > Hi Timo, hi all, > > we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had > configured it in a test environemnt like described in > "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) > > There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" > configuration. All actions on first level directories ".INBOX", ."Sent" > or ".Archives" are working as expected. > > If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an > error message in dovecot log: > Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open > expunge mailbox: Character not allowed in mailbox name: '.' > > What can we do or change to make it work? > sorry didnt tested lazy_plugin yet try http://wiki2.dovecot.org/Plugins/Listescape Examples Allow '.' characters with Maildir++ layout when virtual hierarchy separator is changed to '/': mail_plugins = $mail_plugins listescape namespace private { separator = / inbox = yes } plugin { # The default escape character is '\', but you can change it. # Note that even here the expansion of % takes place, thus you need to # use "%%" if you want to have the % sign as the escape character. #listescape_char = "\\" } or change your general namespace settings to get it work , might be better choice -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 26 15:53:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:53:28 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAC8.9000108@schetterer.org> Am 26.09.2012 13:52, schrieb Maciej Uhlig: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? > > MU > perhaps try http://wiki2.dovecot.org/Plugins/MailLog plugin { # Events to log. Also available: flag_change save mailbox_create mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Also available: flags vsize from subject mail_log_fields = uid box msgid size } -- Best Regards MfG Robert Schetterer From stephan at rename-it.nl Wed Sep 26 15:53:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 26 Sep 2012 14:53:56 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAE4.1010201@rename-it.nl> Op 9/26/2012 1:52 PM, Maciej Uhlig schreef: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? You can use the deliver_log_format setting. The default value is: "msgid=%m: %$" I'm not sure where this is documented, so I took a look at the source code to find out which substitutions exist: static struct var_expand_table static_tab[] = { { '$', NULL, NULL }, { 'm', NULL, "msgid" }, { 's', NULL, "subject" }, { 'f', NULL, "from" }, { 'e', NULL, "from_envelope" }, { 'p', NULL, "size" }, { 'w', NULL, "vsize" }, { '\0', NULL, NULL } }; So, this should work for you: deliver_log_format=msgid=%m size=%p: %$ Regards, Stephan From maciej.uhlig at us.edu.pl Wed Sep 26 16:17:36 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 15:17:36 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062FAE4.1010201@rename-it.nl> References: <5062EC72.5040505@us.edu.pl> <5062FAE4.1010201@rename-it.nl> Message-ID: <50630070.4040000@us.edu.pl> Stephan Bosch: > > So, this should work for you: > > deliver_log_format=msgid=%m size=%p: %$ > Oh yes it did the trick :-) Thank you and best regards, MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 17:15:27 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 16:15:27 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062FA34.7020502@schetterer.org> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> Message-ID: <50630DFF.2050405@TU-Cottbus.De> Hi Robert, >> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >> error message in dovecot log: >> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >> expunge mailbox: Character not allowed in mailbox name: '.' > sorry didnt tested lazy_plugin yet > try > http://wiki2.dovecot.org/Plugins/Listescape It works now with "mail_plugins = $mail_plugins ... lazy_expunge listescape" Thank you for your hint in the right direction. -- Mit freundlichen Gr??en A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ Diese Nachricht wurde aus 100% wiederverwerteten Elektronen erstellt. _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From CMarcus at Media-Brokers.com Wed Sep 26 17:21:07 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 26 Sep 2012 10:21:07 -0400 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50616A32.7040808@thelounge.net> References: <50616A32.7040808@thelounge.net> Message-ID: <50630F53.8060906@Media-Brokers.com> On 2012-09-25 4:24 AM, Reindl Harald wrote: > Am 25.09.2012 10:07, schrieb Voytek Eymont: >> I have a user who occasionally reads, then deletes emails using his phone, >> on his and others email accounts, causing emails deleted on mail server, >> then he says 'oops, can you recover emails I've deleted?' >> >> what's the best way to duplicate all emails to duplicate accounts ? backup >> folders ? so, next time he deletes emails they can be 'recovered' on the >> server ? > configure his phone to leave messages on the server > > if this is the case and he is deleting them manually explain > him that it is NOT your problem if he is too stupid to use > email at all and sooner or later he will learn it Not an option if said user is the boss... -- Best regards, Charles From robert at schetterer.org Wed Sep 26 17:31:41 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 16:31:41 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <506311CD.9060608@schetterer.org> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> Am 25.09.2012 10:07, schrieb Voytek Eymont: >>> I have a user who occasionally reads, then deletes emails using his >>> phone, >>> on his and others email accounts, causing emails deleted on mail server, >>> then he says 'oops, can you recover emails I've deleted?' >>> >>> what's the best way to duplicate all emails to duplicate accounts ? >>> backup >>> folders ? so, next time he deletes emails they can be 'recovered' on the >>> server ? >> configure his phone to leave messages on the server >> >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... > you may do it like this, other soltuions may also thinkable ( depends on your local needs and setup) configure email archive i.e with postfix with i.e always_bcc = address Deliver a copy of all mail to the specified address. In Postfix versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), or pickup(8). sender_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope sender address for an automatic BCC address. This feature is available in Postfix 2.1 and later. recipient_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope recipient address for an automatic BCC address. This feature is available in Postfix 2.1 and later. deliver bc whatever to a second imap account mailbox in a folder set read only via acl i.e with sieve, if you want ,sort by date too etc so if mail was lost ,why ever, the user is able to login into his archive and copy that mail back to his real account -- Best Regards MfG Robert Schetterer From kgc at corp.sonic.net Wed Sep 26 20:34:18 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 10:34:18 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 Message-ID: <50633C9A.5060700@corp.sonic.net> Timo - I upgraded to 2.1.10 on our director servers two nights ago and apart from errors associated with the directors processes restarting everything looked great for ~24 hours until I failed our the real servers last night to update the nfs mount options for the spools. I followed the suggested procedure for each backend server, just run on one of the directors, which seemed to work as expected. doveadm director add x.x.x.x 0 doveadm director flush x.x.x.x The following errors on the directors that started after this went unnoticed until this AM. director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) ... User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. -K From tss at iki.fi Wed Sep 26 20:57:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 20:57:58 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <50633C9A.5060700@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> Message-ID: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> On 26.9.2012, at 20.34, Kelsey Cummings wrote: > The following errors on the directors that started after this went unnoticed until this AM. > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > ... > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? From kgc at corp.sonic.net Wed Sep 26 21:06:33 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 11:06:33 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> Message-ID: <20120926180633.GE80443@corp.sonic.net> On Wed, Sep 26, 2012 at 08:57:58PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 20.34, Kelsey Cummings wrote: > > > The following errors on the directors that started after this went unnoticed until this AM. > > > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > > ... > > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. > > After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? No, there continued to be a mix of both. The pattern seems to look like this. I'll run some stats later but it looks like a pretty significant number of users where affected. 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Wed Sep 26 21:13:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 21:13:26 +0300 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <50630DFF.2050405@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> Message-ID: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> On 26.9.2012, at 17.15, Andreas Kossack wrote: >>> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >>> error message in dovecot log: >>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>> expunge mailbox: Character not allowed in mailbox name: '.' > >> sorry didnt tested lazy_plugin yet >> try >> http://wiki2.dovecot.org/Plugins/Listescape > It works now with > "mail_plugins = $mail_plugins ... lazy_expunge listescape" Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd From h.reindl at thelounge.net Wed Sep 26 21:15:24 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 26 Sep 2012 20:15:24 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <5063463C.9090002@thelounge.net> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... surely even the boss has to understand that there is no technical solution for a social problem - if not: wrong company -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From stsiol at yahoo.co.uk Thu Sep 27 00:19:06 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Wed, 26 Sep 2012 22:19:06 +0100 (BST) Subject: [Dovecot] second dovecot imap server Message-ID: <1348694346.16608.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello again all, I would like to thank everyone for rushing to help me with the dovecot migration issue. I am , currently, reading and studying what has been said by all the good and kind people who responded to? this list. OK, on the same issue,I thought of adding a totally new server with dovecot acting as a secondary/backup server for the mail servicing. Would that be possible ?? What if I manage to make it work; Can I retire the old(er) server ? I would think that this is more on-topic than my last post. If I am out of line again, please respond in private. Thank you all! spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From kgc at corp.sonic.net Thu Sep 27 00:38:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 14:38:37 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <506375DD.3080804@corp.sonic.net> On 09/26/12 11:06, Kelsey Cummings wrote: > No, there continued to be a mix of both. The pattern seems to look like > this. I'll run some stats later but it looks like a pretty significant > number of users where affected. Timo, it looks like the total number of affected users was only about 250 and that most of their erred connections were surrounded by successful sessions. -K From eliezer at ngtech.co.il Thu Sep 27 01:25:07 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 27 Sep 2012 00:25:07 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506380C3.8010903@ngtech.co.il> On 9/24/2012 7:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot It really depends on the raid you have. is it software or hardware raid? if it's software raid it will be more then simple to do it while taking one drive out put a new one in and use let say FINNIX linux from cd\dvd\usb and manage the whole partitioning copying etc from another OS while not harming anything in the old HDDs. you will might need to setup a new partitions manually on the new drive but just make a plan try it on some small VM to make sure the steps you are doing are fine for centos 5.5 and go for it. rsync is your friend!! in this case. someone mentioned here Gentoo which one installation of the system can give you very big backgorund on manual partitioning chrooting and other basic stuff that can help you in the process. Regards, Eliezer > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From amateo at um.es Thu Sep 27 08:52:59 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 27 Sep 2012 07:52:59 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: <5063E9BB.8010104@um.es> I forgot to mention that I'm running dovecot 2.1.9 in my servers. El 26/09/12 12:55, Angel L. Mateo escribi?: > Hello, > > I have a infrastructure using director in front of backend servers. > So I normally run doveadm commands in director servers, not the > backends. The problem I have is when I try to concatenate commands like > this: > > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w > | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} > size.physical mailbox-guid $guid uid $uid; > done > > The problem is that although when I run doveadm search command in > the backend server I correctly get the list of mails, each line with the > mailbox-guid and the uid of the message, when I run the same command in > the director server, format of the list is corrupted and there are lines > that contains just the mailbox-guid and the next the uid (of the > previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 > > so I can't postprocess correctly the output. I think it could be a > bug in the communication between the doveadm client and server. > > PS: As the time of writing this email, I have realized that I could use > fetch command directly. But, anyway, I want to report the problem/bug so > it can be fixed. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From Jost.Krieger+dovecot at rub.de Thu Sep 27 12:08:43 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 11:08:43 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125241.GF2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <20120919125241.GF2010@ruhr-uni-bochum.de> Message-ID: <20120927090843.GW12477@ruhr-uni-bochum.de> On Wed 19 Sep 2012 02:52:41 PM GMT, Jost Krieger wrote: > Oops, that's not what I want! > > How about another command line parameter for lda to suppress the > emergency save? We know exactly when we are trying to save probable > spam. I've tried my luck. Would the attached patch be sensible and welcome? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | -------------- next part -------------- diff -r 88a05f387743 doc/man/dovecot-lda.1.in --- a/doc/man/dovecot-lda.1.in Thu Sep 27 02:55:14 2012 +0300 +++ b/doc/man/dovecot-lda.1.in Thu Sep 27 11:04:56 2012 +0200 @@ -5,7 +5,7 @@ .\"------------------------------------------------------------------------ .SH SYNOPSIS .B dovecot\-lda -.RB [ \-ek ] +.RB [ \-eEk ] [\fB\-a\fP \fIaddress\fP] [\fB\-c\fP \fIconfig_file\fP] [\fB\-d\fP \fIusername\fP] @@ -61,6 +61,10 @@ The default is to send a rejection mail ourself. .\"------------------------------------- .TP +.B \-E +If mail delivery fails, don't try to do an emergency delivery attempt to INBOX. +.\"------------------------------------- +.TP .BI \-f\ envelope_sender Envelope sender address. .\"------------------------------------- diff -r 88a05f387743 src/lda/main.c --- a/src/lda/main.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lda/main.c Thu Sep 27 11:04:56 2012 +0200 @@ -310,7 +310,7 @@ master_service = master_service_init("lda", MASTER_SERVICE_FLAG_STANDALONE | MASTER_SERVICE_FLAG_DONT_LOG_TO_STDERR, - &argc, &argv, "a:d:ef:km:p:r:"); + &argc, &argv, "a:d:eEf:km:p:r:"); memset(&ctx, 0, sizeof(ctx)); ctx.session = mail_deliver_session_init(); @@ -334,6 +334,9 @@ case 'e': stderr_rejection = TRUE; break; + case 'E': + ctx.suppress_emergency_delivery = TRUE; + break; case 'f': /* envelope sender address */ ctx.src_envelope_sender = diff -r 88a05f387743 src/lib-lda/mail-deliver.c --- a/src/lib-lda/mail-deliver.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.c Thu Sep 27 11:04:56 2012 +0200 @@ -402,7 +402,7 @@ ret = mail_deliver_save(ctx, ctx->dest_mailbox_name, 0, NULL, storage_r); } - if (ret < 0 && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { + if (ret < 0 && !ctx->suppress_emergency_delivery && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { /* still didn't work. try once more to save it to INBOX. */ ret = mail_deliver_save(ctx, "INBOX", 0, NULL, storage_r); diff -r 88a05f387743 src/lib-lda/mail-deliver.h --- a/src/lib-lda/mail-deliver.h Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.h Thu Sep 27 11:04:56 2012 +0200 @@ -49,6 +49,7 @@ struct var_expand_table *var_expand_table; bool tried_default_save; + bool suppress_emergency_delivery; bool saved_mail; bool save_dest_mail; /* Delivery failed because user is out of quota / disk space */ From davide.marchi at mail.cgilfe.it Thu Sep 27 12:22:47 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Thu, 27 Sep 2012 11:22:47 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) Message-ID: <50641AE7.6040201@mail.cgilfe.it> Hi to all, sorry in advance for my poor english, this is the first time that i wrote to a list if i make mistake .... excuseme. My problem is this: i have dovecot 2.1.8 installed and functioning from 2 years one week ago i have installed crm114 for my last spam detection filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" My mail system is qmail that through .qmail default correctly tag with X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 X-CRM114-Status: GOOD ( 28.64 ) headers dovecot.conf is as follow: plugin { . . . antispam_signature = X-CRM114-CacheID antispam_verbose_debug = 1 antispam_debug_target = stderr antispam_backend = crm114 antispam_signature_missing = error antispam_trash = Trash antispam_spam = SPAM antispam_unsure = UNSURE antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = --fileprefix=/opt/crm114/ . . . } protocol imap { . . . mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = $mail_plugins quota imap_quota mail_log antispam . . . } I have a sieve default compiled script that automatically put mails in SPAM folder or UNSURE folder if respectively in message hedere there is the evaluation of crm114 (UNSURE or SPAM). When i move mails incorrectly tagged by crm (for example from SPAM to my inbox or from UNSURE to SPAM) thunderbird (company official mail client ver 15.01) go in error with the following message: "Operation failed over folder 'UNSURE'. Server for account davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 binary.." I dont'know where is my mistake if someone can help me is much appreciated Thank you -- */Davide Marchi /Teorema Ferrara Srl /(Tel: /**/+39 0532 783161)/**/ (Fax: +/**/39 0532 783368/**/)/**//**/ /**//**/Davide.Marchi at mail.cgilfe.it /CONFIDENZIALITA? *** Questo messaggio, inclusi gli eventuali allegati, ? indirizzato solo ai destinatari e pu? contenere informazioni riservate e confidenziali. Se avete ricevuto il messaggio senza esserne un destinatario, siete pregati di non utilizzare, copiare o distribuire questo messaggio o i suoi allegati. Se avete ricevuto il messaggio per errore, siete pregati di cancellarlo dal vostro sistema, assieme a tutti gli allegati, e di informare immediatamente il mittente,sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. *CONFIDENTIALITY*** This message and any attachments are intended only for the use of the addressee(s) and may contain information that is privileged and/or confidential. If the reader of the message is not the intended recipient(s) or an authorized representative of the intended recipient(s), please do not use, copy, distribute this email or its attachments or take action based on them. If you have received this communication in error, please notify us immediately by email and delete the message and any attachments from your system. *Teorema FerraraSrl - Via Spronello, 7 - 44121 Ferrara Italia - P. IVA 01244490387** P**S**tampa questo messaggio solo se veramente necessario, pensa alla natura! - Please consider whether it is necessary to print this e-mail* From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 14:41:15 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 13:41:15 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox Message-ID: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Hello. After we switched the format from mbox to mdbox on our dovecot (version 2.1.9) servers last week, some of our users were missing the subfolders of their inboxes. So far we thought that with the mbox format it is not possible to have mail folders containing both messages and subfolders. Now we know that there is one exception from this rule: the inbox. With mbox format dovecot supports subfolders of the inbox, and a few of our users actually detected and used this feature! Internally the messages are stored in /var/mail/user or in a file named 'inbox' and the subfolders are in the directory 'INBOX'. (The OS is case sensitiv.) The problem was that doveadm and dsync do not support subfolders of the inbox and therefore these folders did not get synchronized from our old mbox server to the new mdbox dovecot server. The command doveadm -o mail_location=mbox:~user123/Mail mailbox list -u user123 '*' did not show any subfolder of INBOX, whereas the the following with a wrong mail_location showed that there are subfolders of INBOX doveadm -o mail_location=mbox:~user123 mailbox list -u user123 '*' Finally we could import the missing subfolders one after one in the mdbox server with commands like this: doveadm import -u user123 mbox:~user123/Mail "" mailbox INBOX/subfolder all It would be nice if doveadm could be changed to not overlook subfolders of the inbox with mbox format. Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From Jost.Krieger+dovecot at rub.de Thu Sep 27 15:13:13 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 14:13:13 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Message-ID: <20120927121131.GC12477@ruhr-uni-bochum.de> On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > After we switched the format from mbox to mdbox on our dovecot > (version 2.1.9) servers last week, some of our users were missing > the subfolders of their inboxes. So far we thought that with the > mbox format it is not possible to have mail folders containing both > messages and subfolders. Now we know that there is one exception > from this rule: the inbox. With mbox format dovecot supports > subfolders of the inbox, and a few of our users actually detected > and used this feature! Please see the thread "No status for INBOX/* in mbox NS (2.1.6)" there's a patch from Timo. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 15:56:07 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 14:56:07 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927121131.GC12477@ruhr-uni-bochum.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> <20120927121131.GC12477@ruhr-uni-bochum.de> Message-ID: <20120927145607.89971mht64oiixnk@webmail.hrz.uni-giessen.de> Am Do, 27 Sep 2012 schrieb Jost Krieger: > On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > >> After we switched the format from mbox to mdbox on our dovecot >> (version 2.1.9) servers last week, some of our users were missing >> the subfolders of their inboxes. So far we thought that with the >> mbox format it is not possible to have mail folders containing both >> messages and subfolders. Now we know that there is one exception >> from this rule: the inbox. With mbox format dovecot supports >> subfolders of the inbox, and a few of our users actually detected >> and used this feature! > > Please see the thread > "No status for INBOX/* in mbox NS (2.1.6)" > there's a patch from Timo. Thanks for the info. This patch seems to be in dovecot 2.1.10, we made our format switch from mbox to mdbox with version 2.1.9. Obviously one week too early ;-). Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From blevi.linux at gmail.com Thu Sep 27 16:25:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Thu, 27 Sep 2012 16:25:34 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <506453CE.7000608@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > Hi Timo I just want to ask you: this issue is still in your task list? If you need more debug information please tell me how can I give you. Thanks, Levi From humberto.pedroso at gmail.com Thu Sep 27 17:36:09 2012 From: humberto.pedroso at gmail.com (Humberto A. Pedroso) Date: Thu, 27 Sep 2012 11:36:09 -0300 Subject: [Dovecot] Deleted to trash plugin Message-ID: Hello, Does anybody are using deleted_to_trash plugin with Dovecot 2.1.9? I'm trying to compile and I'm with some difficulties. I tested with dovecot 2.0 and it works very well. -- Humberto From bob at computerisms.ca Thu Sep 27 19:37:20 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 09:37:20 -0700 Subject: [Dovecot] sieve vacation Message-ID: <1348763840.533.187.camel@worklian> Hello, I set up pigeonhole on a server and am using roundcube's sieverules to create a set of rules. When using the vacation feature, I can see in the logs that dovecot is sending the vacation response, but the recipient never receives it. What method does dovecot use to send the response? Does it use the sendmail binary, does it use the mail command, does it connect to some smtp server somewhere, does it use qmail-inject if it finds it? I find no evidence it is using any of these methods on my server, but I am not sure if one of these methods is failing or if there is some place else I should be looking for the missing vacation response... Thank you for any suggestions... -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From robert at schetterer.org Thu Sep 27 19:52:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 18:52:29 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348763840.533.187.camel@worklian> References: <1348763840.533.187.camel@worklian> Message-ID: <5064844D.2000502@schetterer.org> Am 27.09.2012 18:37, schrieb Bob Miller: > the logs that dovecot is sending the vacation response, but the > recipient never receives it. show the logs study http://wiki2.dovecot.org/LDA Common configuration The settings are listed in the example conf.d/15-lda.conf file. The important settings are: postmaster_address is used as the From: header address in bounce mails hostname is used in generated Message-IDs and in Reporting-UA: header in bounce mails sendmail_path is used to send mails. Note that the default is /usr/sbin/sendmail, which doesn't necessarily work the same as /usr/lib/sendmail. Alternatively you can use submission_host to send mails via the specified SMTP server. auth_socket_path specifies the UNIX socket to auth-userdb where LDA can lookup userdb information when -d parameter is used. See below how to configure Dovecot to configure the socket. -- Best Regards MfG Robert Schetterer From bob at computerisms.ca Thu Sep 27 20:12:51 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 10:12:51 -0700 Subject: [Dovecot] sieve vacation In-Reply-To: <5064844D.2000502@schetterer.org> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> Message-ID: <1348765971.533.189.camel@worklian> Robert, > The settings are listed in the example conf.d/15-lda.conf file. The > important settings are: > > sendmail_path is used to send mails. Note that the default is > /usr/sbin/sendmail, which doesn't necessarily work the same as > /usr/lib/sendmail. > > Alternatively you can use submission_host to send mails via the > specified SMTP server. This is the missing piece of information I wasn't finding. Thank you very much... From robert at schetterer.org Thu Sep 27 21:11:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 20:11:00 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348765971.533.189.camel@worklian> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> <1348765971.533.189.camel@worklian> Message-ID: <506496B4.4000508@schetterer.org> Am 27.09.2012 19:12, schrieb Bob Miller: > Robert, > > >> The settings are listed in the example conf.d/15-lda.conf file. The >> important settings are: >> >> sendmail_path is used to send mails. Note that the default is >> /usr/sbin/sendmail, which doesn't necessarily work the same as >> /usr/lib/sendmail. >> >> Alternatively you can use submission_host to send mails via the >> specified SMTP server. > > This is the missing piece of information I wasn't finding. Thank you > very much... > youre welcome Bob *g -- Best Regards MfG Robert Schetterer From jbates at brightok.net Thu Sep 27 23:30:18 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 27 Sep 2012 15:30:18 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused Message-ID: <5064B75A.7060307@brightok.net> Version: 2.1.9 Connecting directly to the proxy server on the LMTP port, issue lhlo, mail from, rcpt to, data. After the . closing of data, it just hangs there. The logs on the proxy server show: Sep 27 19:55:12 proxy1 dovecot: lmtp(9398): Connect from 69.8.2.71 Sep 27 19:55:24 proxy1 dovecot: lmtp(9398): Error: lmtp client: connect(lmtp, 7025) failed: Connection refused root 9398 1102 0 19:53 ? 00:00:00 dovecot/lmtp The proxy server is hard coded (not director in this case). I purposefully shut it down to see how the system would handle failures. Unfortunately, it appears that the process just hangs there. For fun, I tried a separate protocol lmtp option in the config to try proxy_timeout. It made no difference. Missing some error handling? protocol lmtp { passdb { driver = static args = proxy=y host=%s nopassword=y proxy_timeout=120 } } Jack From Andreas.Kossack at TU-Cottbus.De Fri Sep 28 10:25:55 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Fri, 28 Sep 2012 09:25:55 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> Message-ID: <50655103.5090205@TU-Cottbus.De> Hi Timo, hi all, >>>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>>> expunge mailbox: Character not allowed in mailbox name: '.' >> >>> try >>> http://wiki2.dovecot.org/Plugins/Listescape >> It works now with >> "mail_plugins = $mail_plugins ... lazy_expunge listescape" > > Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd Now it works lika a charm without "listescape" "mail_plugins = $mail_plugins ... lazy_expunge" Thank you Timo... -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ From jbates at brightok.net Fri Sep 28 19:37:50 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 11:37:50 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5064B75A.7060307@brightok.net> References: <5064B75A.7060307@brightok.net> Message-ID: <5065D25E.1030507@brightok.net> On 9/27/2012 3:30 PM, Jack Bates wrote: > Version: 2.1.9 > > Connecting directly to the proxy server on the LMTP port, issue lhlo, > mail from, rcpt to, data. After the . closing of data, it just hangs > there. The logs on the proxy server show: > Looking at dovecot's proxy code. Calls to connect_connect_ip do not return an error when the server isn't there. This may be expected, but it is what determines failure/OK in the LMTP code. The log entry is performed by lmtp_client_wait_connect callback. In the standard client-common-auth.c code, the error returned for a failure is 1) syslog error is reported by proxy_wait_connect() callback 2) error to client is returned by an error in proxy_input() In the lmtp code, we issue a "250 2.1.5 OK". Right after, the lmtp_client_wait_connect() callback is called and issues the connection refused error. However, we've already approved the rcpt. I am not sure which method we really want in the lmtp proxy. Should it be failing at the initial rcpt command or after the data command. Currently, though I haven't checked yet, we don't appear to be detecting the input failure and handling the data command correctly since we've already allowed the rcpt to proceed without a backend proxy available. Jack From jbates at brightok.net Fri Sep 28 20:44:21 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 12:44:21 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065D25E.1030507@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> Message-ID: <5065E1F5.4010506@brightok.net> On 9/28/2012 11:37 AM, Jack Bates wrote: > On 9/27/2012 3:30 PM, Jack Bates wrote: >> Version: 2.1.9 >> >> Connecting directly to the proxy server on the LMTP port, issue lhlo, >> mail from, rcpt to, data. After the . closing of data, it just hangs >> there. The logs on the proxy server show: > I am not sure which method we really want in the lmtp proxy. Should it > be failing at the initial rcpt command or after the data command. > Currently, though I haven't checked yet, we don't appear to be > detecting the input failure and handling the data command correctly > since we've already allowed the rcpt to proceed without a backend > proxy available. Further testing shows that the connection fails are normally handled after the DATA block. It works perfectly if you only fail one recipient. If all recipients are failed, the code locks up instead of telling the client that they are all failed. Now to find out where between lmtp_client_fail() and some other part of the code, we aren't treating it right. :( Jack From tss at iki.fi Fri Sep 28 21:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 21:59:52 +0300 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 In-Reply-To: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> References: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Message-ID: <43585454-3155-4B89-B5FD-2D2ACA80E562@iki.fi> On 22.9.2012, at 19.11, Clemens Schrimpe wrote: > Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: > > sudo /usr/libexec/StartupItemContext dovecot > > (again: all this was working fine under 2.0.15) > > Now with 2.1.10, when I manually invoke dovecot with just > > sudo dovecot > > Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. I don't really know about mach contexts or how they're supposd to work.. > What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? Not much I think. I just looked at the diff between 2.0 and 2.1 and don't really see anything I could blame. If you have time you could try bisecting with mercurial (basically try different versions from hg) to isolate the change that broke it. I don't really have time to debug this.. From tss at iki.fi Fri Sep 28 22:01:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:01:13 +0300 Subject: [Dovecot] Dovecot Clustering with dsync over ssh In-Reply-To: References: Message-ID: On 23.9.2012, at 10.56, weber at zackbummfertig.de wrote: > i only found an old post of tiemo where he announced that he will do an wiki article > how to setup dovecot cluster with dsync over ssh. > anyone knows if this artcile exists and where? > > i would like to setup a dovecot cluster with 2 machines. There are some mailing list posts about how to do it. Anyway I don't advertise it much yet because v2.1's dsync isn't really quite good enough to do it well. v2.2 dsync should work well. From tss at iki.fi Fri Sep 28 22:02:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:02:42 +0300 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <9952E585-05B8-4418-A3BD-0ED0F0A84165@iki.fi> On 26.9.2012, at 0.34, Marc Perkel wrote: > Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? v2.1 has passdb imap From jbates at brightok.net Fri Sep 28 23:12:44 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:12:44 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065E1F5.4010506@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> Message-ID: <506604BC.6050503@brightok.net> On 9/28/2012 12:44 PM, Jack Bates wrote: > On 9/28/2012 11:37 AM, Jack Bates wrote: >> On 9/27/2012 3:30 PM, Jack Bates wrote: >>> Version: 2.1.9 >>> >>> Connecting directly to the proxy server on the LMTP port, issue >>> lhlo, mail from, rcpt to, data. After the . closing of data, it just >>> hangs there. The logs on the proxy server show: >> > Further testing shows that the connection fails are normally handled > after the DATA block. It works perfectly if you only fail one > recipient. If all recipients are failed, the code locks up instead of > telling the client that they are all failed. > > Now to find out where between lmtp_client_fail() and some other part > of the code, we aren't treating it right. :( It appears that the callbacks for the proxy connections are handled in the ioloop stuff. If you have at least one valid proxy session open, then proxy_try_finish will get called and all the replies are sent. The problem appears to be if all backend proxy sessions are bad (ie single recipient, proxy server down), there is nothing for ioloop to callback to. proxy_try_finish will never get called, and we end up locking up at epoll_wait as there are no events for us to process. Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. Jack From jbates at brightok.net Fri Sep 28 23:29:11 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:29:11 -0500 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <506604BC.6050503@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> Message-ID: <50660897.6040008@brightok.net> On 9/28/2012 3:12 PM, Jack Bates wrote: > > Code needs to be written to handle the special case of us not having > any proxy callbacks as they are all bad. > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 *************** *** 300,303 **** --- 300,304 ---- lmtp_client_send(conn->client, conn->data_input); lmtp_client_send_more(conn->client); } + lmtp_proxy_try_finish(proxy); } ie, call lmtp_proxy_try_finish once. If all is bad, this will wrap us up. If we have valid proxies, it'll probably not finish and we'll return to waiting on callbacks. I'm not sure of any blocking restrictions. I just know it works. mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) 250 2.0.0 CYKfHcsHZlBcCAAALhEySA Saved mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 250 2.0.0 EYKfHcsHZlBcCAAALhEySA Saved Jack From CMarcus at Media-Brokers.com Sun Sep 30 16:02:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 30 Sep 2012 09:02:03 -0400 Subject: [Dovecot] Log NAT IP address? Message-ID: <506842CB.8080501@Media-Brokers.com> Hi Timo/everyone, Currently we are logging the remote IP, but is there a way to show the IP address that the NAT connection is coming from? The reason I ask is, we are changing ISPs, and I would like to see in the logs when an external connection is coming from our OLD ISP connection, and when it is coming through our new one. We have a Watchguard firewall, and I have both External connections setup and working, and have just pointed our DNS records to the new public IP, and would like to be able to see which WAN connection/IP they are coming from. Thanks, -- Best regards, Charles From jbates at brightok.net Sun Sep 30 17:33:14 2012 From: jbates at brightok.net (Jack Bates) Date: Sun, 30 Sep 2012 09:33:14 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068582A.6030507@brightok.net> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. > > We have a Watchguard firewall, and I have both External connections > setup and working, and have just pointed our DNS records to the new > public IP, and would like to be able to see which WAN connection/IP > they are coming from. You could bind 2 internal IP Addresses to the server and have each NAT translation go to a different internal IP. Jack From dovec at nettrust.co.nz Sat Sep 1 01:42:32 2012 From: dovec at nettrust.co.nz (Michael) Date: Sat, 01 Sep 2012 10:42:32 +1200 Subject: [Dovecot] dovecotadm error Message-ID: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> When I use the following command: doveadm expunge -A mailbox Junk savedbefore 28d I get: doveadm(root): Error: User listing returned failure In the mail log file it says: dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table 'system.users' doesn't exist (using built-in default iterate_query: SELECT username, domain FROM users) However this is setup: dovecot-sql.conf: iterate_query = SELECT username AS user FROM accounts though I have had to comment the following from dovecot.conf: #iteratedb { # args = /usr/local/etc/dovecot/dovecot-sql.conf # driver = sql #} because it errors on dovecot start up. I have already spent a bit of time on the Dovecot website and Google trying to find an answer. I want to get the expires plugin working. Any leads please? dovecot -n # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.29.6-rt24-smp i686 Slackware 13.1.0 auth_master_user_separator = * auth_mechanisms = plain login dict { expire = mysql:/usr/local/etc/dovecot/dovecot-dict-expire.conf quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 mail_access_groups = dovecot mail_location = Maildir:%h/Maildir mail_plugins = " expire" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Drafts autocreate2 = Junk autocreate3 = Sent autocreate4 = Trash autosubscribe = Drafts autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Trash expire = Junk expire_dict = proxy::expire quota = dict:user::proxy::quotadict sieve = %h/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/before.sieve sieve_dir = %h/sieve sieve_global_dir = /var/lib/dovecot/sieve/global/ } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service dict { unix_listener dict { group = dovecot mode = 0660 } } service imap-login { inet_listener imap { address = DELETED } } service managesieve-login { executable = /usr/local/libexec/dovecot/managesieve-login } service managesieve { executable = /usr/local/libexec/dovecot/managesieve } ssl_cert = I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) disconnect in IDLE and inactivity time outs. I'm using only IMAP with two users. Both users access the server from multiple devices, phone, tablet and desktop computer. I had previously set the max_userip_connections to 20 and rarely got the following limits exceeded message. That's probably being triggered when we are both home and using all of our devices on the same network. I've now changed it to 40 and we'll see how that does. The other is the IDLE and inactivity timeouts. I don't really understand how IDLE works and I couldn't find an inactivity timeout setting to play with. There are at this second 32 imap or imap-login processes running with only two phones and one desktop running. That's a few more than before the upgrade but not a lot. --------------------- Dovecot Begin ------------------------ Dovecot limits exceeded: max_userip_connections: penny from 216.240.58.138 to 216.240.58.140: 56 Time(s) Dovecot disconnects: Disconnected in IDLE: 485 Time(s) Inactivity: 343 Time(s) no auth attempts: 14 Time(s) no reason: 21 Time(s) Everything seems to be working OK so maybe I should just not worry about it? Thanks, knute... # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_anonymous_username = anonymous auth_cache_negative_ttl = 1 hours auth_cache_size = 0 auth_cache_ttl = 1 hours auth_debug = no auth_debug_passwords = no auth_default_realm = auth_failure_delay = 2 secs auth_first_valid_uid = 500 auth_gssapi_hostname = auth_krb5_keytab = auth_last_valid_uid = 0 auth_master_user_separator = auth_mechanisms = plain login auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 60 default_internal_user = dovecot default_login_user = dovenull default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins disable_plaintext_auth = yes dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_proxy_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 first_valid_gid = 1 first_valid_uid = 500 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = imap_idle_notify_interval = 2 mins imap_logout_format = bytes=%i/%o imap_max_line_length = 64 k import_environment = TZ info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/lib/dovecot listen = *, :: lmtp_proxy = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = syslog log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = Dovecot ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib/dovecot/modules mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no passdb { args = deny = no driver = pam master = no pass = no } pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = " imap" quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs inet_listener { address = port = 0 ssl = no } privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener director-userdb { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 0 ssl = no } inet_listener pop3s { address = port = 0 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = Hi, I've not found the answer to this question anywhere - please forgive me if I overlooked. I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net From acrow at integrafin.co.uk Sat Sep 1 12:42:16 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Sat, 01 Sep 2012 10:42:16 +0100 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: <5041D878.3090609@integrafin.co.uk> On 01/09/12 09:10, David Anderson wrote: > Hi, > > I've not found the answer to this question anywhere - please forgive > me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are > hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same > way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and > logwatch can be noisy and it's easy to overlook. > > My motivation: for some reason my POP3 listener was hanging, and I had > to restart dovecot (this is 2.0.9). But I was not aware - the process > was still there, it was accepting connections (but not doing anything > useful with them) and was actually managing to log the fact that the > process limit was being hit (as more and more connections came in). If > I could have been alerted I could have restarted dovecot earlier. > > Many thanks, > David > I believe that nagios or icinga could do this for you with a log analyser plugin. http://exchange.nagios.org/directory/Plugins/Log-Files Cheers Alex From leamhall at gmail.com Sat Sep 1 12:53:15 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 05:53:15 -0400 Subject: [Dovecot] Testing process? Message-ID: Morning all! Total newbie question. I'm learning Dovecot and will be plugging in a few things and testing it end to end. What test structures or tools are available for making sure the install is correct and the parameters are taken? Thanks! Leam -- Mind on a Mission From robert at schetterer.org Sat Sep 1 13:30:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 01 Sep 2012 12:30:18 +0200 Subject: [Dovecot] Testing process? In-Reply-To: References: Message-ID: <5041E3BA.3090100@schetterer.org> Am 01.09.2012 11:53, schrieb leam hall: > Morning all! Total newbie question. I'm learning Dovecot and will be > plugging in a few things and testing it end to end. What test structures or > tools are available for making sure the install is correct and the > parameters are taken? > > Thanks! > > Leam > > describe what you mean "install is correct" as there are unnumbered ways to setup dovecot usally you found your errors in the logs, for testing use verbose logging so first plan "what setup" you want to goal, reading dovecot sites , faqs, examples, archives anyway perhaps use some virt. machine for testing multiple stuff -- Best Regards MfG Robert Schetterer From kprprl at gmail.com Sat Sep 1 15:25:19 2012 From: kprprl at gmail.com (PARTH MONGA) Date: Sat, 1 Sep 2012 17:55:19 +0530 Subject: [Dovecot] Testing process? In-Reply-To: <5041E3BA.3090100@schetterer.org> References: <5041E3BA.3090100@schetterer.org> Message-ID: netstat -tulpn port listening on 110(pop) 143(imap) confirms the same and you can also do cat /var/log/dovecot.log rgds p On Sat, Sep 1, 2012 at 4:00 PM, Robert Schetterer wrote: > Am 01.09.2012 11:53, schrieb leam hall: > > Morning all! Total newbie question. I'm learning Dovecot and will be > > plugging in a few things and testing it end to end. What test structures > or > > tools are available for making sure the install is correct and the > > parameters are taken? > > > > Thanks! > > > > Leam > > > > > > describe what you mean "install is correct" > as there are unnumbered ways to setup dovecot > > usally you found your errors in the logs, for testing use verbose logging > > so first plan "what setup" you want to goal, reading dovecot sites , > faqs, examples, archives > > anyway perhaps use some virt. machine for testing multiple stuff > -- > Best Regards > MfG Robert Schetterer > From leamhall at gmail.com Sat Sep 1 17:19:07 2012 From: leamhall at gmail.com (leam hall) Date: Sat, 1 Sep 2012 10:19:07 -0400 Subject: [Dovecot] Testing process? In-Reply-To: References: <5041E3BA.3090100@schetterer.org> Message-ID: > > describe what you mean "install is correct" > > as there are unnumbered ways to setup dovecot > > > > usally you found your errors in the logs, for testing use verbose logging > > > > so first plan "what setup" you want to goal, reading dovecot sites , > > faqs, examples, archives > > > > anyway perhaps use some virt. machine for testing multiple stuff > > -- > > Best Regards > > MfG Robert Schetterer > > > Robert, et al, thank you for reminding me that I need to share what I'm thinking! I've been bouncing it around in my own head but didn't really write enough to communicate. There are a lot of things i need to learn about Dovecot, so first is how to send something to it and then see where it's logged or what actions are taken. My work uses Dovecot and I need to be able to use, test, and understand it. So maybe the first question is, is there a test framework for Dovecot itself or is it better to set up with postfix or something and just route mail through it? Thanks! Leam -- Mind on a Mission From david at wordshell.net Sat Sep 1 21:08:53 2012 From: david at wordshell.net (David Anderson) Date: Sat, 01 Sep 2012 21:08:53 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <1346490929.10359.16.camel@tardis> References: <5041C2FD.1040108@wordshell.net> <1346490929.10359.16.camel@tardis> Message-ID: <50424F35.2040105@wordshell.net> Hi, Thanks. I used monit, and will code something up of that kind - but I was curious as to whether dovecot had something built in; external monitoring is a somewhat blunter tool, as you don't get to know exactly why there was a problem unless you also parse the logs. Tricky to investigate this kind of issue - it has only happened once, and I had a few hundred people trying to log in, so no time to start debugging! But if it recurs I'll be back to ask for guidance... Many thanks, David -- WordShell - WordPress fast from the CLI - www.wordshell.net On 01/09/12 12:15, Noel Butler wrote: > Seen similar, we use mon? for network monitoring, our mon pop3 script > is modified to not only connect, but login, if it cant the trigger > becomes active and if fails again at next check, it alerts engineers > by sms. > > Though, I would investigate the core issue, Timo "apparently" fixed > that hung session stuff in 2.0, it was common in 0.x and 1.x series. > > Cheers > > 1: ( > http://sourceforge.net/projects/mon/files/mon/ ) > > > From ml at smtp.fakessh.eu Sun Sep 2 04:59:42 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 02 Sep 2012 03:59:42 +0200 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler Message-ID: <1346551182.14379.1.camel@localhost> Hi Timo in the present work would be possible to walk to another folder for deleted_to_trash please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From me at junc.org Sun Sep 2 08:30:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 07:30:40 +0200 Subject: [Dovecot] =?utf-8?q?How_Converting/Using_courier-userdb_Files_=3F?= In-Reply-To: <5040C15D.4000303@eurodata.de> References: <5040C15D.4000303@eurodata.de> Message-ID: <421fa10c33e339153cb5532da61e57b2@junc.org> Den 2012-08-31 15:51, Dirk Caspari skrev: > how can i convert/use the Original userdb-Files from courier > Installation ? > The files are looking like a "Special"-DB-Format and are not > readable. it can be added if gdbm is supported in dovecot auth backedn, even convert the gdbm db to sqlite will be suported in shells, when its converted lets say to sqlite then its a matter of define how dovecot use it its a one time run From tss at iki.fi Sun Sep 2 17:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 17:28:28 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120831084233.GE3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: On 31.8.2012, at 11.42, Jost Krieger wrote: > One of our users likes to delete attachments from his mails. He is using > roundcube and we installed a plugin for that. Does the plugin change the mbox file directly? > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error. From eddy.ilg at univation.de Sun Sep 2 18:28:26 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:28:26 +0200 Subject: [Dovecot] Special Folder Mocking Message-ID: <50437B1A.8000808@univation.de> Dear Dovecot List, probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. A simple question: Isn't it possible to advertise one imap folder with different names, depending on the client? E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? Best regards, Eddy Ilg From tss at iki.fi Sun Sep 2 18:32:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:08 +0300 Subject: [Dovecot] Mailbox aliases Message-ID: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: .. mailbox Sent { auto = create special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent alias_for = Sent } .. If "Sent Messages" doesn't exist: - DELETE Sent works as usual - RENAME Sent works as usual If "Sent Messages" is created: - it's symlinked to Sent - DELETE "Sent messages" deletes the symlink, not the contents - RENAME "Sent messages" isn't allowed - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. Thoughts? The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Sun Sep 2 18:32:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 18:32:37 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> On 2.9.2012, at 18.28, Eddy Ilg|univation wrote: > Dear Dovecot List, > > > probably everyone knows the problem that different clients use different names for "Sent", "Trash" and so on. I was just in the middle of writing the previous mail about this :) > A simple question: > Isn't it possible to advertise one imap folder with different names, depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? No. From eddy.ilg at univation.de Sun Sep 2 18:50:12 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Sun, 02 Sep 2012 17:50:12 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> Message-ID: <50438034.4060604@univation.de> > I was just in the middle of writing the previous mail about this :) Already thought you replied and changed the subject of my post ;) >> A simple question: >> Isn't it possible to advertise one imap folder with different names, depending on the client? >> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? > No. But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? A similar thing is "imap_client_workarounds = tb-extra-mailbox-sep". If what I suggested is implementable then what speaks against it? If I have some time I might try writing a patch... Or maybe a dovecot developer can implement it, who better knows the code. Best, Eddy From tss at iki.fi Sun Sep 2 19:39:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 2 Sep 2012 19:39:46 +0300 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50438034.4060604@univation.de> References: <50437B1A.8000808@univation.de> <7CE87A48-17C2-4C49-A85C-179FF8B72477@iki.fi> <50438034.4060604@univation.de> Message-ID: On 2.9.2012, at 18.50, Eddy Ilg|univation wrote: >> I was just in the middle of writing the previous mail about this :) > Already thought you replied and changed the subject of my post ;) >>> A simple question: >>> Isn't it possible to advertise one imap folder with different names, depending on the client? >>> E.g. if the client is Outlook, show the "Sent" folder as "Sent Elements" (translated from german) and if the client is Thunderbird, show the "Sent" folder as "Sent"? >> No. > But would it be theoretically possible? I.e. do the clients report themselves to dovecot correctly? Even theoretically it's not something I want to consider. It would cause more trouble than gain. (Some clients identify themselves with ID extension, many don't (e.g. Outlook). Some clients might be unreliably detected using the IMAP command tags they use, but that could identify other clients wrong. Different localizations of clients use different mailbox names.) From dovecot at knutejohnson.com Sun Sep 2 20:29:17 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:29:17 -0700 Subject: [Dovecot] Strange messages? Message-ID: <5043976D.7040806@knutejohnson.com> The following messages show up in the output of a script that does some backup. This too is new with the upgrade to Ubuntu server 12.04.1 and dovecot 2.0.19. Anybody know what would generate these sorts of messages? Where or what would cause them? WARN: Duplicate profile 'Dovecot POP3', using last found WARN: Duplicate profile 'Dovecot Secure POP3', using last found WARN: Duplicate profile 'Dovecot IMAP', using last found WARN: Duplicate profile 'Dovecot Secure IMAP', using last found Thanks, -- Knute Johnson From dovecot at knutejohnson.com Sun Sep 2 20:30:18 2012 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sun, 02 Sep 2012 10:30:18 -0700 Subject: [Dovecot] New log entries with 2.0.19? In-Reply-To: <5041436E.7000208@knutejohnson.com> References: <5041436E.7000208@knutejohnson.com> Message-ID: <504397AA.7040300@knutejohnson.com> A couple of days later and it is back to normal. Never mind. knute... On 8/31/2012 4:06 PM, Knute Johnson wrote: > I just upgraded my Ubuntu server from 11.10 to 12.04.1 and that upgraded > Dovecot to 2.0.19. For some reason I'm getting a lot more (5 times) > disconnect in IDLE and inactivity time outs. I'm using only IMAP with > two users. Both users access the server from multiple devices, phone, > tablet and desktop computer. I had previously set the > max_userip_connections to 20 and rarely got the following limits > exceeded message. That's probably being triggered when we are both home > and using all of our devices on the same network. I've now changed it > to 40 and we'll see how that does. > > The other is the IDLE and inactivity timeouts. I don't really > understand how IDLE works and I couldn't find an inactivity timeout > setting to play with. There are at this second 32 imap or imap-login > processes running with only two phones and one desktop running. That's > a few more than before the upgrade but not a lot. > > --------------------- Dovecot Begin ------------------------ > > Dovecot limits exceeded: > max_userip_connections: penny from 216.240.58.138 to > 216.240.58.140: 56 Time(s) > > Dovecot disconnects: > Disconnected in IDLE: 485 Time(s) > Inactivity: 343 Time(s) > no auth attempts: 14 Time(s) > no reason: 21 Time(s) > > Everything seems to be working OK so maybe I should just not worry about > it? > > Thanks, > > knute... > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_anonymous_username = anonymous > auth_cache_negative_ttl = 1 hours > auth_cache_size = 0 > auth_cache_ttl = 1 hours > auth_debug = no > auth_debug_passwords = no > auth_default_realm = > auth_failure_delay = 2 secs > auth_first_valid_uid = 500 > auth_gssapi_hostname = > auth_krb5_keytab = > auth_last_valid_uid = 0 > auth_master_user_separator = > auth_mechanisms = plain login > auth_realms = > auth_socket_path = auth-userdb > auth_ssl_require_client_cert = no > auth_ssl_username_from_cert = no > auth_use_winbind = no > auth_username_chars = > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ > auth_username_format = > auth_username_translation = > auth_verbose = no > auth_verbose_passwords = no > auth_winbind_helper_path = /usr/bin/ntlm_auth > auth_worker_max_count = 30 > base_dir = /var/run/dovecot > config_cache_size = 1 M > debug_log_path = > default_client_limit = 1000 > default_idle_kill = 60 > default_internal_user = dovecot > default_login_user = dovenull > default_process_limit = 100 > default_vsz_limit = 256 M > deliver_log_format = msgid=%m: %$ > dict_db_config = > director_doveadm_port = 0 > director_mail_servers = > director_servers = > director_user_expire = 15 mins > disable_plaintext_auth = yes > dotlock_use_excl = yes > doveadm_allowed_commands = > doveadm_password = > doveadm_proxy_port = 0 > doveadm_socket_path = doveadm-server > doveadm_worker_count = 0 > first_valid_gid = 1 > first_valid_uid = 500 > hostname = > imap_capability = > imap_client_workarounds = > imap_id_log = > imap_id_send = > imap_idle_notify_interval = 2 mins > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > import_environment = TZ > info_log_path = > instance_name = dovecot > last_valid_gid = 0 > last_valid_uid = 0 > lda_mailbox_autocreate = no > lda_mailbox_autosubscribe = no > lda_original_recipient_header = > libexec_dir = /usr/lib/dovecot > listen = *, :: > lmtp_proxy = no > lmtp_save_to_detail_mailbox = no > lock_method = fcntl > log_path = syslog > log_timestamp = "%b %d %H:%M:%S " > login_access_sockets = > login_greeting = Dovecot ready. > login_log_format = %$: %s > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > login_trusted_networks = > mail_access_groups = > mail_attachment_dir = > mail_attachment_fs = sis posix > mail_attachment_hash = %{sha1} > mail_attachment_min_size = 128 k > mail_cache_fields = flags > mail_cache_min_mail_count = 0 > mail_chroot = > mail_debug = no > mail_fsync = optimized > mail_full_filesystem_access = no > mail_gid = > mail_home = > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_log_prefix = "%s(%u): " > mail_max_keyword_length = 50 > mail_max_lock_timeout = 0 > mail_max_userip_connections = 10 > mail_never_cache_fields = imap.envelope > mail_nfs_index = no > mail_nfs_storage = no > mail_plugin_dir = /usr/lib/dovecot/modules > mail_plugins = > mail_privileged_group = > mail_save_crlf = no > mail_temp_dir = /tmp > mail_uid = > mailbox_idle_check_interval = 30 secs > mailbox_list_index_disable = no > maildir_copy_with_hardlinks = yes > maildir_stat_dirs = no > maildir_very_dirty_syncs = no > master_user_separator = > mbox_dirty_syncs = yes > mbox_dotlock_change_timeout = 2 mins > mbox_lazy_writes = yes > mbox_lock_timeout = 5 mins > mbox_min_index_size = 0 > mbox_read_locks = fcntl > mbox_very_dirty_syncs = no > mbox_write_locks = dotlock fcntl > mdbox_preallocate_space = no > mdbox_rotate_interval = 0 > mdbox_rotate_size = 2 M > mmap_disable = no > passdb { > args = > deny = no > driver = pam > master = no > pass = no > } > pop3_client_workarounds = > pop3_enable_last = no > pop3_fast_size_lookups = no > pop3_lock_session = no > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_save_uidl = no > pop3_uidl_format = %08Xu%08Xv > postmaster_address = > protocols = " imap" > quota_full_tempfail = no > recipient_delimiter = + > rejection_reason = Your message to <%t> was automatically rejected:%n%r > rejection_subject = Rejected: %s > sendmail_path = /usr/sbin/sendmail > service anvil { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = anvil > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 1 > protocol = > service_count = 0 > type = anvil > unix_listener anvil-auth-penalty { > group = > mode = 0600 > user = > } > unix_listener anvil { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service auth-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = auth -w > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener auth-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service auth { > chroot = > client_limit = 4096 > drop_priv_before_exec = no > executable = auth > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener auth-client { > group = > mode = 0600 > user = > } > unix_listener auth-login { > group = > mode = 0600 > user = $default_internal_user > } > unix_listener auth-master { > group = > mode = 0600 > user = > } > unix_listener auth-userdb { > group = > mode = 0600 > user = > } > unix_listener login/login { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service config { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = config > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = config > unix_listener config { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service dict { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dict > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dict { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service director { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = director > extra_groups = > fifo_listener login/proxy-notify { > group = > mode = 00 > user = > } > group = > idle_kill = 4294967295 secs > inet_listener { > address = > port = 0 > ssl = no > } > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener director-admin { > group = > mode = 0600 > user = > } > unix_listener director-userdb { > group = > mode = 0600 > user = > } > unix_listener login/director { > group = > mode = 00 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service dns_client { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dns-client > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dns-client { > group = > mode = 0666 > user = > } > unix_listener login/dns-client { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service doveadm { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = doveadm-server > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener doveadm-server { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service imap-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = imap-login > extra_groups = > group = > idle_kill = 0 > inet_listener imap { > address = > port = 143 > ssl = no > } > inet_listener imaps { > address = > port = 993 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service imap { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = > unix_listener login/imap { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ipc { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = ipc > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener ipc { > group = > mode = 0600 > user = > } > unix_listener login/ipc-proxy { > group = > mode = 0600 > user = $default_login_user > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service lmtp { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = lmtp > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = lmtp > service_count = 0 > type = > unix_listener lmtp { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service log { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = log > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = log > user = > vsz_limit = 18446744073709551615 B > } > service pop3-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = pop3-login > extra_groups = > group = > idle_kill = 0 > inet_listener pop3 { > address = > port = 0 > ssl = no > } > inet_listener pop3s { > address = > port = 0 > ssl = yes > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service pop3 { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = pop3 > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = pop3 > service_count = 1 > type = > unix_listener login/pop3 { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service ssl-params { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = ssl-params > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = startup > unix_listener login/ssl-params { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > shutdown_clients = yes > ssl = yes > ssl_ca = > ssl_cert = ssl_cert_username_field = commonName > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > ssl_client_cert = > ssl_client_key = > ssl_key = ssl_key_password = > ssl_parameters_regenerate = 168 > ssl_verify_client_cert = no > submission_host = > syslog_facility = mail > userdb { > args = > driver = passwd > } > valid_chroot_dirs = > verbose_proctitle = no > verbose_ssl = no > version_ignore = no > protocol imap { > imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID > ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND > UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 ESEARCH > ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS > mail_max_userip_connections = 40 > } > -- Knute Johnson From p at state-of-mind.de Sun Sep 2 23:32:24 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sun, 2 Sep 2012 22:32:24 +0200 Subject: [Dovecot] Special Folder Mocking In-Reply-To: <50437B1A.8000808@univation.de> References: <50437B1A.8000808@univation.de> Message-ID: <20120902203221.GB18740@state-of-mind.de> Eddy, * Eddy Ilg|univation : > Dear Dovecot List, > > > probably everyone knows the problem that different clients use > different names for "Sent", "Trash" and so on. > > A simple question: > Isn't it possible to advertise one imap folder with different names, > depending on the client? > E.g. if the client is Outlook, show the "Sent" folder as "Sent > Elements" (translated from german) and if the client is Thunderbird, > show the "Sent" folder as "Sent"? the answer is the IMAP extension "SPECIAL USE". We sponsored SPECIAL USE in Dovecot and in Thunderbird. Timo implemented it in Dovecot autumn 2011 and Ben Bucksch added SPECIAL USE support in Mozilla shortly after. Sadly Mozilla has decided to turn Thunderbird down - i.e. hand it over to some 'community' - and I am not sure our contribution it will be released as Mozilla claims they will not add any new features anymore. IF they would you could tell Dovecot to name the SPECIAL USE folders like Outlook expects them and Thunderbird to map its folder on top. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Mon Sep 3 00:01:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 02 Sep 2012 23:01:16 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <5043C91C.2070608@schetterer.org> Am 02.09.2012 17:32, schrieb Timo Sirainen: > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. Hi Timo, funny, yesterday i had problems with this, i am using virtual plugin setup with pop3 also disallow i.e imap Sent/Trash/Drafts download via imap, but allow Junk imap folder to be downloaded via pop3 cause i have a global sieve rule storing, that spam tagged mail is going direct in Junk folder, that usally cant be overided by users. This should avoid redirecting spam tagged mail via sieve by users in prime. This works nice. My users use all kind of os and mail clients typical ISP Setup Default webmail Horde Imp layout is configured for thunderbirds default imap folder layout, also thunderbird is announced to be the supported mail client via the faq help site, not because its the best client appearing, only for ,that it is installable in all major os, so debug is possible widly In the help site and pictured welcome pdf mail there are setups shown for all major mail clients, as outlook, thunderbird , winmail. Not for Apple cause i havent got the chance to make screenshots from it yet. I also have active sync via z-push so setup in android mail app is on the help site too. So far so good. With Problems that i.e outlook is very different in handling imap folder setup through its versions As more and more people going to use mobile, stuff get more complicated As one user uses k9mail android vers 4.11 ( this is recent ), which cant do imap special use yet, so a i.e german Sent Folder ( Gesendet ) is created auto by the k9 client, same user does pop3 via outlook so the problem appeared that he send mail with copy in the "Gesendet" from K9 mail android folder ,but this Folder wasnt disallowed in the pop3 virtual layout , so a loop appeared. I had now to disallow folder "Gesendet/Papierkorb/Entw?rfe" for pop3. Upcomming Versions K9 Mail 4.2 can do imap special use, and it works by testing Only small real World example. I investigated little, and found neither Thunderbird Outlook or Apple is supporting imap special use yet ( seems its on the road for thunderbird ), perhaps someone knows more about this. My meaning to this theme is, its simply teribble for 21 century mail that there is no recommended default imap folder layout for all mailclients, imap special use looks promising , hopefully it will wide spread Goal: Configure mail should be done auto only with emailadress and password including default imap folder layout and match trough all major mail imap clients. So example wiki site for configure dovecot relate to major imap clients and their versions behave in default imap folder layout and support of imapspecial use would be extrem helpfull, in fact yesterday i searched about that. -- Mfg Best Regards Robert Schetterer From me at junc.org Mon Sep 3 00:22:40 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 02 Sep 2012 23:22:40 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> Message-ID: <0fa581317d82ab7f071329317125bc55@junc.org> Den 2012-08-24 07:18, Voytek Eymont skrev: > I want to offer 'plain' unsecure pop/imap access, just for temp. > testing purpose, do I simply comment out > ssl = required plain is not related to ssl enabled, here i have disabled pop3 imap, so now here i only supports ssl / tls connections better set listen to localhost or another rfc1918 ip From h.reindl at thelounge.net Mon Sep 3 00:32:34 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 02 Sep 2012 23:32:34 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <5043C91C.2070608@schetterer.org> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> <5043C91C.2070608@schetterer.org> Message-ID: <5043D072.4020604@thelounge.net> Am 02.09.2012 23:01, schrieb Robert Schetterer: > Goal: > Configure mail should be done auto only with emailadress and password > including default imap folder layout and match trough all major mail > imap clients. > > So example wiki site for configure dovecot relate to major imap clients > and their versions behave in default imap folder layout and support of > imapspecial use would be extrem helpfull, in fact yesterday i searched > about that. the real problem are stupid clients at all the folders just could called "Sent" on the server and the client can display whatever it wants, this even works if you take some minutes to configure all your clients to use the same folder and the apple is coming: eahc different version of their clients is creating different sent folders - after update MacOSX it happens sometimes that it creates a new incarnation on the servr and spit new messages in the new one instead the over months used existing and if a users owns different apple-devices he ends up in many incarnations * Sent * Sent Messages * Sent Messages (acount name) * Gesendet and this is only the example of ONE apple-only-user you can imagine what happens using different other clients and smartphone-types this is nothing that can be fixed on the server side the idiotic client developers should take care of the different possible existing special folders and use any of them if existing instead create a new one -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 01:31:11 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:31:11 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0fa581317d82ab7f071329317125bc55@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> Message-ID: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From raubvogel at gmail.com Mon Sep 3 01:38:53 2012 From: raubvogel at gmail.com (Mauricio) Date: Sun, 2 Sep 2012 22:38:53 +0000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> Message-ID: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> I thought you could specify ssl to be wanted but not required, but I might be thinking on client side. Sent via iPhone -----Original Message----- From: "Voytek Eymont" Sender: dovecot-bounces at dovecot.orgDate: Mon, 3 Sep 2012 08:31:11 To: Subject: Re: [Dovecot] Disable security for testing? On Mon, September 3, 2012 7:22 am, Benny Pedersen wrote: > Den 2012-08-24 07:18, Voytek Eymont skrev: > >> I want to offer 'plain' unsecure pop/imap access, just for temp. >> testing purpose, do I simply comment out ssl = required > > plain is not related to ssl enabled, here i have disabled pop3 imap, so > now here i only supports ssl / tls connections > > better set listen to localhost or another rfc1918 ip Benny, thanks, I think I'm using incorrect terminology, I'm sorry, basically, I wanted to temporarily remove the Dovecot default requirement for pop and imap access to be secure/encrypted (I have some mail clients from 'old' server attempting to access email on new server, and failing due to only-encrypted access on new server) and, yes, by commenting out " ssl = required" it allowed the 'not properly configured for new server' client to access -- Voytek From voytek at sbt.net.au Mon Sep 3 01:44:16 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 08:44:16 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <710bc96f1513f58dd51935702626803e.squirrel@sbt.net.au> On Mon, September 3, 2012 8:38 am, Mauricio wrote: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. Sent via iPhone I'm happy to have it as a default requirement, seems like a reasonable requirement, this was just for some temp testing/verification etc -- Voytek From me at junc.org Mon Sep 3 02:26:20 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:26:20 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> Message-ID: <0535537af33849715f4b3efd97569d4d@junc.org> Den 2012-09-03 00:38, Mauricio skrev: > I thought you could specify ssl to be wanted but not required, but I > might be thinking on client side. the point is ? > Sent via iPhone i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port 110/143 anymore From h.reindl at thelounge.net Mon Sep 3 02:32:57 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 01:32:57 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <0535537af33849715f4b3efd97569d4d@junc.org> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> Message-ID: <5043ECA9.5010005@thelounge.net> Am 03.09.2012 01:26, schrieb Benny Pedersen: > i dont have a iphone, but my custommers do say its working with ssl/tls and my own dovecot does not listen on port > 110/143 anymore why? STARTTLS is 110/143 per definition! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From me at junc.org Mon Sep 3 02:40:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:40:56 +0200 Subject: [Dovecot] =?utf-8?q?Disable_security_for_testing=3F?= In-Reply-To: <5043ECA9.5010005@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: Den 2012-09-03 01:32, Reindl Harald skrev: > STARTTLS is 110/143 per definition! EOD From adi at ddns.com.au Mon Sep 3 02:41:14 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 09:41:14 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> Message-ID: <20120903094114.2a75835d@adi.ddns.local> On Fri, 31 Aug 2012 17:11:07 +0300 Timo Sirainen wrote: > > The POP/IMAP part is working fine. What I'm trying to do is to use > > Dovecot SASL implementation in Postfix to do SMTP authentication in > > a similar manner. The problem I have with my current configuration > > is that SMTP authentication succeeds if only the username matches, > > because password forwarding works if the authentication succeeds > > with any given password, as documented at > > http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy > > Dovecot has no SMTP proxy (currently). And anyway Postfix doesn't use > SMTP to do authentication, Postfix authenticates using Dovecot's > internal protocol, which replies that Postfix should do the proxying, > which it of course doesn't do. Yes, I know and that's exactly what I was trying to do: use Dovecot authentication method in Postfix to authenticate the user. Because of the fact that Dovecot doesn't do SMTP authentication, I was thinking of a way of using its authentication service by getting the SMTP login credentials from the backend POP/IMAP server. The request may sound a bit unusual, but in our case it makes sense. For our setup we currently run: - a farm of backend SMTP/POP/IMAP servers that are hosting the mailboxes and where the user credentials are managed. They are running Courier IMAP. - a group of SMTP/POP/IMAP proxies. These proxies are currently replicating the login credentials from the backend servers and the routing to the backends using a local database. Perdition is currently the POP/IMAP proxy, but having it replaced with Dovecot would help us in getting the password forwarding to the backends running, which means we wouldn't need to store the credentials on the proxy, only the user->host routing entries. Are there any plans to have Dovecot authentication service to do SMTP authentication against IMAP or POP3 proxy provided information? This, of course, means we'd have the authentication result tied to the response of the backend IMAP/pop3 server. > > My question is, given the above: is there a way to get SMTP > > authentication properly in this scenario? > > Make Postfix authenticate against the backend Dovecot server. You'll > need to setup service auth { inet_listener } to some port for it. The POP/IMAP backends are running Courier IMAP, as I've just mentioned and due to the existing hosting environment it's very unlikely to replace it with something else. -- Adi Pircalabu, System Administrator Discount Domain Name Services Pty Ltd, a Total Internet Company PO Box 887, Hawthorn Vic 3122, Australia, T +61 3 9815 6868 Ask me about cloud hosting services From tss at iki.fi Mon Sep 3 02:47:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 02:47:48 +0300 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <20120903094114.2a75835d@adi.ddns.local> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: On 3.9.2012, at 2.41, Adi Pircalabu wrote: > Are there any plans to have Dovecot authentication service to do SMTP > authentication against IMAP or POP3 proxy provided information? This, > of course, means we'd have the authentication result tied to the > response of the backend IMAP/pop3 server. I don't see any clean way of doing that. >>> My question is, given the above: is there a way to get SMTP >>> authentication properly in this scenario? >> >> Make Postfix authenticate against the backend Dovecot server. You'll >> need to setup service auth { inet_listener } to some port for it. > > The POP/IMAP backends are running Courier IMAP, as I've just mentioned > and due to the existing hosting environment it's very unlikely to > replace it with something else. Maybe use IMAP authentication as the backend? pam_imap at least can do that. Or you can already also use Dovecot v2.1's passdb imap to do this, pretty much equivalent to pam_imap. From ml at smtp.fakessh.eu Mon Sep 3 02:55:20 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 01:55:20 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: <32bbc94eb73b728d0a4646c6d7a3b67d@smtp.fakessh.eu> I currently work this plugin with several folders nice a documentation include Le 2012-09-02 03:59, ml a ?crit?: > Hi Timo > > in the present work would be possible to walk to another folder for > deleted_to_trash > > please answer me -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From me at junc.org Mon Sep 3 02:55:21 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 03 Sep 2012 01:55:21 +0200 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> Message-ID: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Den 2012-09-03 01:47, Timo Sirainen skrev: > Maybe use IMAP authentication as the backend? pam_imap at least can > do that. Or you can already also use Dovecot v2.1's passdb imap to do > this, pretty much equivalent to pam_imap. drop pam, and use saslauthd with remote imap, or setup cyrus sasl with sql/ldap/whatever one needs, just in case one more ask why i did not use dovecot :) From tss at iki.fi Mon Sep 3 03:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 03:18:22 +0300 Subject: [Dovecot] possible work plugin deleted_to_trash for otrher foler In-Reply-To: <1346551182.14379.1.camel@localhost> References: <1346551182.14379.1.camel@localhost> Message-ID: On 2.9.2012, at 4.59, ml wrote: > in the present work would be possible to walk to another folder for > deleted_to_trash I suppose, with some code modifications. But I'd prefer if nobody used this plugin (or anything equivalent to its functionality). From ml at smtp.fakessh.eu Mon Sep 3 03:26:54 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 03 Sep 2012 02:26:54 +0200 Subject: [Dovecot] =?utf-8?q?possible_work_plugin_deleted=5Fto=5Ftrash_for?= =?utf-8?q?_otrher_foler?= In-Reply-To: References: <1346551182.14379.1.camel@localhost> Message-ID: <303d5f8de294c07f547603b58a4aec85@smtp.fakessh.eu> Le 2012-09-03 02:18, Timo Sirainen a ?crit?: > On 2.9.2012, at 4.59, ml wrote: > >> in the present work would be possible to walk to another folder for >> deleted_to_trash > > I suppose, with some code modifications. But I'd prefer if nobody > used this plugin (or anything equivalent to its functionality). until the it works very well as it is with an inclusion in dovecot.conf. and why not use it I have folders in my email that I am happy with this empty nice plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From adi at ddns.com.au Mon Sep 3 04:55:20 2012 From: adi at ddns.com.au (Adi Pircalabu) Date: Mon, 3 Sep 2012 11:55:20 +1000 Subject: [Dovecot] [Postfix] SASL Auth. using Dovecot with password forwarding proxy configuration In-Reply-To: <0e7ab168e17bd41dcc6638de74ac5981@junc.org> References: <20120831125120.37a3f7cf@adi.ddns.local> <13A83420-05D7-4DBF-A09B-2D2EE1FE9F48@iki.fi> <20120903094114.2a75835d@adi.ddns.local> <0e7ab168e17bd41dcc6638de74ac5981@junc.org> Message-ID: <20120903115520.218c8190@adi.ddns.local> On Mon, 03 Sep 2012 01:55:21 +0200 Benny Pedersen wrote: > Den 2012-09-03 01:47, Timo Sirainen skrev: > > > Maybe use IMAP authentication as the backend? pam_imap at least can > > do that. Or you can already also use Dovecot v2.1's passdb imap to > > do this, pretty much equivalent to pam_imap. I had a look at pam_imap and had to fix the spec file to get it built for CentOS 6 64bit. For the record, I had to add the following in the %build section: export CFLAGS="%{optflags} -fPIC" export CXXFLAGS="%{optflags} -fPIC" However: > drop pam, and use saslauthd with remote imap, or setup cyrus sasl > with sql/ldap/whatever one needs Yep, saslauthd with rimap appears to be exactly what I need. > just in case one more ask why i did not use dovecot :) The password forwarding feature is one of the reasons I started looking into Dovecot. Thanks to you both for your help. -- Adi Pircalabu, System Administrator From amateo at um.es Mon Sep 3 09:48:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 08:48:54 +0200 Subject: [Dovecot] exclude a folder for lazy expunge (move to trash) In-Reply-To: <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> References: <503491C4.7030208@um.es> <712399DF-2E5C-4BCF-BF4B-3C8A919D4F9B@iki.fi> Message-ID: <504452D6.8@um.es> El 31/08/12 15:23, Timo Sirainen escribi?: > On 22.8.2012, at 11.01, Angel L. Mateo wrote: > >> I'm configuring lazy expunge plugin to act as a kind of backup of my user's emails. >> >> I'm getting a little problem because a lot of my users use "move to trash" behaviour in their email client (most of them are using thunderbird which has no option to hide deleted messages and is a hassle working with mark as deleted with it). The problem combining move to trash with lazy expunge is that messages are stored in the expunge namespace twice, first when it was deleted from the original folder and the second when is deleted from the trash. >> >> So... is there any way to exclude trash folder from lazy expunge plugin, so mails are stored in the expunge namespace only when they are deleted from their original folder and not from the trash? > > Not right now. Maybe it could be automatically done when SPECIAL-USE is enabled and the deletion is from a \Trash mailbox. > >> Another idea... We are purging trash folders with a cron running "doveadm purge" commands. Is there any way to run this command disabling lazy expunge plugin? > > doveadm purge shouldn't affect lazy expunge, but you can disable lazy_expunge by using something like doveadm -o mail_plugins=everything,except,lazy_expunge expunge .. > Sorry, I meant "doveadm expunge" (I always confuse these two commands) which does affect lazy expunge -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From alec at alec.pl Mon Sep 3 10:02:13 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 03 Sep 2012 09:02:13 +0200 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: <504455F5.7000504@alec.pl> On 09/02/2012 05:32 PM, Timo Sirainen wrote: > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > Thoughts? Yes. It should solve some problems, but I think that only wide-use of SPECIAL-USE is a solution for this. Also.... I don't use current version of dovecot, but from reading this list I remember that SPECIAL-USE implementation in dovecot is read-only. What I'd like to see is possibility to set special-use flags by the client - no server configuration required. It means, when user connects to his mailbox for the first time and special folders doesn't exist, the client (supporting SPECIAL-USE) can create folders according to its configuration and set their special-use flags. Or at any time when it find that special folder doesn't exist. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From zucca at systemschmiede.com Mon Sep 3 10:01:51 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Mon, 03 Sep 2012 09:01:51 +0200 Subject: [Dovecot] Imap Ghost folder In-Reply-To: <292te8ob8kv8@mids.svenhartge.de> References: <50341D40.9090001@systemschmiede.com> <50353D32.2080401@systemschmiede.com> <5038B2C5.3090408@systemschmiede.com> <4CE233FA-2954-4C39-9F79-13C12ADFDCF2@iki.fi> <5038CB71.5010808@systemschmiede.com> <503C818A.4010203@systemschmiede.com> <503DE2D9.3080300@systemschmiede.com> <20120829204814.GA9265@daniel.localdomain> <503F0B77.5030206@systemschmiede.com> <504072FA.8020606@systemschmiede.com> <56D9DD4B-ED96-47C6-96F4-322D6907508C@iki.fi> <5040C32C.2020706@systemschmiede.com> <85E5DEFA-D38E-441E-BFE9-64FC820069E8@iki.fi> <5040C789.8000502@systemschmiede.com> <292te8ob8kv8@mids.svenhartge.de> Message-ID: <504455DF.4020202@systemschmiede.com> >> hm, but 2.1.9 is not in backports yet, is it? > And will not be for some time. 2.1.9 is not even in Unstable and even if > it were, it would not migrate to testing (and be an eligible backporting > target) due to the freeze before the Wheezy release. So, no workaround for this? Noone can reproduce it using my configuration? Very strange.. Thanks Sascha. From amateo at um.es Mon Sep 3 11:41:24 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 10:41:24 +0200 Subject: [Dovecot] doveadm command to block a user? Message-ID: <50446D34.5000702@um.es> Hello, Is there any command to be able to lock imap access to a specific user? With doveadm kick I could close his connections, but I want to avoid future ones too and just for imap. Any way to do it? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tseveendorj at gmail.com Mon Sep 3 12:10:14 2012 From: tseveendorj at gmail.com (tseveendorj) Date: Mon, 03 Sep 2012 17:10:14 +0800 Subject: [Dovecot] dovecot sasl + postfix issue Message-ID: <504473F6.5010205@gmail.com> Hello, I'm trying to configure postfix + Dovecot SASL for user authenticated mail relay. I set following configuration on postfix queue_directory = /var/spool/postfix smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_authenticated_header = yes smtpd_sasl_security_options = noanonymous smtpd_sasl_local_domain = $myhostname broken_sasl_auth_clients = yes smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination Following are dovecot -n # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS auth_mechanisms = plain login digest-md5 base_dir = /var/run/dovecot/ mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r when I see telnet localhost 25 eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 beastie ESMTP Postfix (Ubuntu) ehlo beastie 250-beastie 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN quit 221 2.0.0 Bye Connection closed by foreign host. there is no 250-AUTH How do I fix this ? From Bill at knoxvillechristian.org Mon Sep 3 12:36:32 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Mon, 03 Sep 2012 05:36:32 -0400 Subject: [Dovecot] dovecot sasl + postfix issue In-Reply-To: <504473F6.5010205@gmail.com> References: <504473F6.5010205@gmail.com> Message-ID: <50447A20.7060900@knoxvillechristian.org> You should get AUTH after you STARTTLS. Bill On 9/3/2012 5:10 AM, tseveendorj wrote: > Hello, > > I'm trying to configure postfix + Dovecot SASL for user authenticated > mail relay. > > I set following configuration on postfix > > queue_directory = /var/spool/postfix > smtpd_sasl_auth_enable = yes > smtpd_sasl_type = dovecot > smtpd_sasl_path = private/auth > smtpd_sasl_authenticated_header = yes > smtpd_sasl_security_options = noanonymous > smtpd_sasl_local_domain = $myhostname > broken_sasl_auth_clients = yes > smtpd_recipient_restrictions = reject_unknown_sender_domain, > reject_unknown_recipient_domain, reject_unauth_pipelining, > permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination > > Following are dovecot -n > > # 2.0.19: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-29-generic x86_64 Ubuntu 12.04.1 LTS > auth_mechanisms = plain login digest-md5 > base_dir = /var/run/dovecot/ > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > passdb { > driver = pam > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap pop3 sieve > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = ssl_cipher_list = > ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM > ssl_key = userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 10 > } > protocol pop3 { > mail_max_userip_connections = 10 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to <%t> was automatically rejected:%n%r > > > > when I see telnet localhost 25 > > eberx at beastie:/etc/dovecot/conf.d$ telnet localhost 25 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > 220 beastie ESMTP Postfix (Ubuntu) > ehlo beastie > 250-beastie > 250-PIPELINING > 250-SIZE 10240000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > quit > 221 2.0.0 Bye > Connection closed by foreign host. > > there is no 250-AUTH > > How do I fix this ? From h.reindl at thelounge.net Mon Sep 3 13:05:10 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:05:10 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> Message-ID: <504480D6.1000900@thelounge.net> Am 03.09.2012 01:40, schrieb Benny Pedersen: > Den 2012-09-03 01:32, Reindl Harald skrev: > >> STARTTLS is 110/143 per definition! > > EOD oh yeah this is the right attitude * open thunderbird * go to server settings * switch between STARTTLS/SSL * look how the default port changes for SMTP port 465 is even deprecated and you should use 587 (submission) with STATTLS for endusers but do what you want if you like support-calls as example from customers sitting in a network where most outgoing ports are blocked (including 993 and 995) while 143 is open, yes i have seen this from customers sitting in gonvernment networks but hey, for you this is EOD because you know it better SMTP STARTTLS: 587 SMTP over SSL: 465 POP3 STARTTLS: 110 POP3 over SSL: 995 IMAP STARTTLS: 143 IMAP over SSL: 993 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Mon Sep 3 13:15:27 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 3 Sep 2012 20:15:27 +1000 Subject: [Dovecot] Disable security for testing? In-Reply-To: <504480D6.1000900@thelounge.net> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> Message-ID: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> > but hey, for you this is EOD because you know it better > > SMTP STARTTLS: 587 > SMTP over SSL: 465 > POP3 STARTTLS: 110 > POP3 over SSL: 995 > IMAP STARTTLS: 143 > IMAP over SSL: 993 in my limited experience, when I tried to set Outlook Express clients with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got it on 993) question: is it because Outlook Express does SSL only, but not TLS? (or is it simply because it's crappy application ?) thanks, enquiring minds need to know, -- Voytek From h.reindl at thelounge.net Mon Sep 3 13:19:16 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 03 Sep 2012 12:19:16 +0200 Subject: [Dovecot] Disable security for testing? In-Reply-To: <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <50448424.7000804@thelounge.net> Am 03.09.2012 12:15, schrieb Voytek Eymont: >> but hey, for you this is EOD because you know it better >> >> SMTP STARTTLS: 587 >> SMTP over SSL: 465 >> POP3 STARTTLS: 110 >> POP3 over SSL: 995 >> IMAP STARTTLS: 143 >> IMAP over SSL: 993 > > > in my limited experience, when I tried to set Outlook Express clients with > my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on 995; > (I think..? I struck same with IMAP, couldn't get ssl/tls on 143, but, got > it on 993) it's a microsoft problem > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) because it is a crappy application you see htunderbird perfectly switching between STARTTLS/SSL ports you can configure each webmail i know to use STARTTLS you can configure even Apple Mail for both usecases -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From amateo at um.es Mon Sep 3 14:16:35 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 03 Sep 2012 13:16:35 +0200 Subject: [Dovecot] Syntax for doveadm auth cache Message-ID: <50449193.8080101@um.es> Hello, What is the correct syntax of doveadm auth cache command? According to command line help: usage: doveadm [-Dv] [-f ] [] altmove [-u |-A] [-S ] [-r] auth [-a ] [-x ] [] auth cache but when I try to run doveadm auth cache, then I'm asked for a password, so I think the command is trying to authenticate cache user. Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): doveadm auth cache flush 4 cache entries flushed So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From sven at svenhartge.de Mon Sep 3 14:31:18 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 3 Sep 2012 13:31:18 +0200 Subject: [Dovecot] finding messages deleted before timespec Message-ID: <4934p48so7v8@mids.svenhartge.de> Hi! I've been pondering about doveadm-search-query(7) and trying to formulate a search query for some time now and it seems there is no way to search for messages, which have been marked DELETED before a certain time. I want to use this to expunge all DELETED messages which have been in that state for 24h or longer. For example a cronjob runs at midnight to expunge those messages but a user deletes one just 10 minutes before that, this message should not be deleted until the day after. This is my first time diving deeper into IMAP search queries so I have the nagging feeling I am missing something important here. Gr??e, Sven. -- Sigmentation fault. Core dumped. From daniel at dhag.net Mon Sep 3 16:32:55 2012 From: daniel at dhag.net (Daniel Funke) Date: Mon, 03 Sep 2012 15:32:55 +0200 Subject: [Dovecot] dovecot proxy ssl-parameter from database? Message-ID: <5044B187.6000609@dhag.net> Hi List, I want to migrate my IMAP-users from an old Courier-IMAP-Server to a new dovecot-Server. The old server supports IMAP without and with TLS. The new one only with TLS. To users should be migrated one by one with changing some client parameters. To leave the servername of the imap server I want to use the dovecot proxy during migration. the proxy uses a sql db to look for the old or new host for every user. So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) I tried with http://wiki2.dovecot.org/UserDatabase/ExtraFields and returned a field "userdb_ssl" ("required" for host dovecot, "yes" for host courier). But it doesn't work. A migrated user can still login without tls to the proxy and the dovecot backend. Any ideas? Daniel Here is my doveconf -n # 2.1.9: /opt/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login default_process_limit = 500 disable_plaintext_auth = no listen = * managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /opt/dovecot/etc/dovecot/sql.conf driver = sql } protocols = imap pop3 service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3s { port = 0 } } ssl_cert = References: <6fbbd67b-dc1e-4875-826f-e646751d6278@email.android.com> <0fa581317d82ab7f071329317125bc55@junc.org> <88dd4f99ef67b71ae319dad5dff750c4.squirrel@sbt.net.au> <516509515-1346625535-cardhu_decombobulator_blackberry.rim.net-629804954-@b14.c1.bise6.blackberry> <0535537af33849715f4b3efd97569d4d@junc.org> <5043ECA9.5010005@thelounge.net> <504480D6.1000900@thelounge.net> <89610731dc7216c70a627aaa5344f96f.squirrel@geko.sbt.net.au> Message-ID: <20120903160119.06de7fcd@zetkin.int.primekey.se> On Mon, 3 Sep 2012 20:15:27 +1000 "Voytek Eymont" wrote: > in my limited experience, when I tried to set Outlook Express clients > with my Dovecot 2, I couldn't get to do ssl/tls on port 110, only on > 995; (I think..? I struck same with IMAP, couldn't get ssl/tls on > 143, but, got it on 993) > > question: > is it because Outlook Express does SSL only, but not TLS? > (or is it simply because it's crappy application ?) The ports 143 and 993 are used in different way when you want to have TLS. In case of 143, you begin the communication in unencrypted mode, then tell the server that you want to initiate a TLS handshake (you literally send a STARTTLS string to it). After that both client and server start negotiating the TLS connection (and hopefully they manage to do it :). With the port 993 the expected/standard behavior is that you begin the TLS handshake right upon establishing the connection. Normally the servers will not accept any unencrypted IMAP command on it (in other words, you don't send STARTTLS to the server, the server expects you to start the handshake right away). Similar concept is implemented for POP3, SMTP, LDAP, XMPP, and many others... Btw, SSL and TLS are kind of the same things (or to be more precise when people are talking about it they refer to the same thing, unless they're crypto techies which are using the terminology correctly :) Best regards P.S. It's quite probably a crappy application as well :) -- Branko Majic Jabber: branko at majic.rs Please use only Free formats when sending attachments to me. ?????? ????? ?????: branko at majic.rs ????? ??? ?? ??????? ?????? ????????? ? ????????? ?????????. From Jost.Krieger+dovecot at rub.de Mon Sep 3 17:11:21 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 3 Sep 2012 16:11:21 +0200 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> Message-ID: <20120903141121.GR3002@ruhr-uni-bochum.de> [Sorry about the personal reply.] On Sun, Sep 02, 2012 at 05:28:28PM +0300, Timo Sirainen wrote: > > On 31.8.2012, at 11.42, Jost Krieger wrote: > > > One of our users likes to delete attachments from his mails. He is using > > roundcube and we installed a plugin for that. > > Does the plugin change the mbox file directly? No, it just uses IMAP. > > > imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) > > This looks like it does. Existing emails must not be changed, IMAP > protocol forbids it (and Dovecot internals don't support it either). > The only safe way to drop attachments is to assign a new UID for the > mail, which basically means saving a new message without attachments > and deleting the old one. The Content-Length header doesn't really > have anything to do with this error. Sorry if I was unclear. This is exactly what the plugin does. It builds a new message, starting with the old message and leaving out the unwanted attchment(s). Then it saves the new message, deletes the old message and finally copies over the flags. I had a look at the handling of "Content-Length" in dovecot and had the impression that a new header was added but the old was not removed (if formally ok) and on reading the first header wins. But I'm not sure, of course. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From kgc at corp.sonic.net Mon Sep 3 21:26:55 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:26:55 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected Message-ID: <5044F66F.3010505@corp.sonic.net> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) I've dropped one of the servers out of the ring to prevent suffering but this is a less than ideal situation. Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 10000 director_mail_servers = 1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 director_servers = a.director.foo b.director.foo disable_plaintext_auth = no doveadm_proxy_port = 1842 login_trusted_networks = 10.0.0.1 mbox_write_locks = fcntl passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 20103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_ca = Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:23:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla2 host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:23:32 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:24:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:05 a.director. a dovecot: director: Warning: Delaying new user requests until ring is synced >Sep 3 09:24:32 a.director. a dovecot: director: Warning: Ring is synced, continuing delayed requests >Sep 3 09:24:41 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:24:41 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:41 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:24:41 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:25:11 b.director. b dovecot: director: Error: User hash 2285697953 is being redirected to two hosts: 10.10.10.39 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689467) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 623192092 is being redirected to two hosts: 10.10.10.76 and 10.10.10.39 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 1683990717 is being redirected to two hosts: 10.10.10.43 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:25:12 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director lef -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From robert at schetterer.org Mon Sep 3 21:32:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 03 Sep 2012 20:32:59 +0200 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <5044F7DB.9080601@schetterer.org> Am 03.09.2012 20:26, schrieb Kelsey Cummings: > I've had 2x director ring up and running with production load on 2.1.8 > with around 10,000 active connections for two weeks and everything has > been working great - until this morning. meanwhile try upgrade to 2.1.9 i cant see big changes there for director which should brake something http://hg.dovecot.org/dovecot-2.1/rev/bc86680293d2 however its better debug with latest patchlevel -- Best Regards MfG Robert Schetterer From eddy.ilg at univation.de Mon Sep 3 21:36:22 2012 From: eddy.ilg at univation.de (Eddy Ilg|univation) Date: Mon, 03 Sep 2012 20:36:22 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. Message-ID: <5044F8A6.7060200@univation.de> Hi, In the documentation of dovecot-antispam it is mentioned that moving mails out of the spam folders trains them as non-spam. I.e. thunderbird automatically deletes old spam messages (after a time limit, i.e. 14 days). Will this also train the old spam messages as non-spam? Cheers, Eddy From kgc at corp.sonic.net Mon Sep 3 21:37:57 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 11:37:57 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F7DB.9080601@schetterer.org> References: <5044F66F.3010505@corp.sonic.net> <5044F7DB.9080601@schetterer.org> Message-ID: <5044F905.60204@corp.sonic.net> On 9/3/2012 11:32 AM, Robert Schetterer wrote: > Am 03.09.2012 20:26, schrieb Kelsey Cummings: >> I've had 2x director ring up and running with production load on 2.1.8 >> with around 10,000 active connections for two weeks and everything has >> been working great - until this morning. > > meanwhile try upgrade to 2.1.9 I did that in the second sentence. ;) One of the servers had a single segfault but this wasn't the original cause and didn't show up until after I was poking. director[30709]: segfault at 40 ip 0000000000405d37 sp 00007fff42af80e0 error 6 in director[400000+f000] -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Mon Sep 3 22:06:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 3 Sep 2012 22:06:49 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. > > There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) Did you try stopping both and then starting them again? That clears up all the state they have. > Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. Directors themselves think they're having trouble connecting to each others.. Annoyingly it doesn't give specific error messages about what happened. I should improve the logging.. If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. From kgc at corp.sonic.net Tue Sep 4 04:58:16 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 03 Sep 2012 18:58:16 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <50456038.4070008@corp.sonic.net> On 9/3/2012 12:06 PM, Timo Sirainen wrote: > Did you try stopping both and then starting them again? That clears up all the state they have. I'm not sure that they were both down when restarting them and will try this tonight. > If the state clearing doesn't help, maybe this has something to do with the OS or the network is really having some issues. I can't rule that out but there are not any signs that there are any hardware, OS or network related issues. Thanks for gettting the ring status into doveadm by the way. At least our monitoring caught this quickly. -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From bmalik at ihlas.net.tr Tue Sep 4 07:15:23 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Mon, 3 Sep 2012 21:15:23 -0700 (PDT) Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <20120814230403.GA11246@daniel.localdomain> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> Message-ID: <1346732123552-37388.post@n4.nabble.com> i am sorry when i login to the server using pop3s, i see in the dovecot log; @4000000050457dd9179c653c pop3-login: Info: Login: user=, method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, session= if i add my mail address into the users-with-forced-encryption file, i would not reach my account. i get an error; @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script denied access to user test at byserver.net whereas i connect to the account via pop3s if i remove my mail address from the users-with-forced-encryption there is no problem. my postlogin_pop3.sh; #!/usr/local/bin/bash if [ "$SECURED" == "secure" ] then :# handle secure connections here else :# handle insecure connections here if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption then printf "* NO [ALERT] You are not allowed to connect without encryption\r\n" exit 0 fi fi exec "$@" ########### there is also no problem related to SSL certificate what can be the issue ? -- View this message in context: http://dovecot.2317879.n4.nabble.com/about-postlogin-in-dovecot2-tp5993p37388.html Sent from the Dovecot mailing list archive at Nabble.com. From bangkokmaco at gmail.com Tue Sep 4 07:21:54 2012 From: bangkokmaco at gmail.com (cc young) Date: Mon, 3 Sep 2012 21:21:54 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting Message-ID: <1346732514245-37389.post@n4.nabble.com> cannot get TB to recognize either pop3/s or imap/s server can connect just fine with: openssl s_client -connect ms1.myserver.net:993 . login ... but trying with TB /var/log/mail.log gets: dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx -- View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html Sent from the Dovecot mailing list archive at Nabble.com. From me at junc.org Tue Sep 4 08:00:35 2012 From: me at junc.org (Benny Pedersen) Date: Tue, 04 Sep 2012 07:00:35 +0200 Subject: [Dovecot] dovecot-antispam and Thunderbird, Outlook, etc. In-Reply-To: <5044F8A6.7060200@univation.de> References: <5044F8A6.7060200@univation.de> Message-ID: Den 2012-09-03 20:36, Eddy Ilg|univation skrev: > In the documentation of dovecot-antispam it is mentioned that moving > mails out of the spam folders trains them as non-spam. I.e. > thunderbird automatically deletes old spam messages (after a time > limit, i.e. 14 days). Will this also train the old spam messages as > non-spam? i hope it depends on dovecot-antispam knows the trashcan or not, but seen from here i think this is really a bug, here i use v1 of dovecot and i dont know if this is changed in v2 dovecot if needs be post plugin config, here i am unsure about this aswell :/ From stsiol at yahoo.co.uk Tue Sep 4 08:59:47 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 06:59:47 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: cc young > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 7:21 > Subject: [Dovecot] thunderbird not connecting > > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: > http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. > Thunderbird is not the best of clients out there. It's ok but not the best. I am using TB on my sites and found out that it has many issues. Can you be a bit more specific ? Are you trying to connect to Secure IMAP ? (993) on dovecot ? In that case, as far as I remember you have to "fool" Thunderbird during the mail account set up process.? It's a bit crazy, I know but it's true. I have documented everything. Let me know where the hickup is and I'll try to get the docs and let you know how to work around this. HTH, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Tue Sep 4 10:31:22 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 08:31:22 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 9:42 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 12:59 PM, Spyros Tsiolis wrote: > > >> >>----- Original Message ----- >>> From: cc young >>> To: dovecot at dovecot.org >>> Cc: >>> Sent: Tuesday, 4 September 2012, 7:21 >>> Subject: [Dovecot] thunderbird not connecting >>> >>> cannot get TB to recognize either pop3/s or imap/s server >>> >>> can connect just fine with: >>> >>> openssl s_client -connect ms1.myserver.net:993 >>> . login ... >>> >>> but trying with TB /var/log/mail.log gets: >>> >>> dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, >>> lip=xxx.xx.xx.xx >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html >>> Sent from the Dovecot mailing list archive at Nabble.com. >>> >> >> >>Thunderbird is not the best of clients out there. >>It's ok but not the best. >> >>I am using TB on my sites and found out that it has many issues. >>Can you be a bit more specific ? >>Are you trying to connect to Secure IMAP ? (993) on dovecot ? >> >>In that case, as far as I remember you have to "fool" Thunderbird >>during the mail account set up process.? >> >>It's a bit crazy, I know but it's true. I have documented everything. >>Let me know where the hickup is and I'll try to get the docs and >>let you know how to work around this. >> >>HTH, >> > > >right - ??trying to connect to Secure IMAP ? (993) on dovecot > > >not in love with thunderbird, but need to access via linux and ms > > >any help / insight would be wonderful > >? Hi again, Taken from my docs . This is the first part for Mozilla Thunderbird setup. I've seen this work both on Win32 systems and linux systems (mainly Linux Mint). There's a part on pop3/smtp. I am sure you can skip that and do your own magic for those two protocols :? Pre-Installation, Wizard Configuration -------------------------------------- During initial Thunderbird startup, the admin is greeted by a welcome window named ?Mail Account Setup?? Click on Cancel Installation ------------ Click on ?Edit? -> ??Account Settings? New window pops up Click on ?Add? New window pops up Fill in ?Description? (e.g. ?Name Surname mailbox?) On ?Server Name? enter the IP address (e.g. 192.168.3.5) On ?Port? enter the SMTP port (e.g. 25) On ?Security and Authentication? ? ?Connection Security? enter none On ?Authentication Method? choose Password transmitted insecurely On ?Username? enter the users' username including the FQDN (e.g. name at domain.gr) Click on OK From ?Edit?, click on ?Account Settings? From ?Account Actions? click on ?Add Mail Account? New windows pops up Enter in the same name for ?Your name? Enter in the same e-mail address for ?Email address? On ?Password? enter in users' password Click on ?Continue? then without waiting click on ?Manual Config? Once ?Manual Config? is clicked, thunderbird will allow the admin to perform additional configuration tasks For ?Incoming? the admin will choose ?IMAP?, Server Hostname will be set to the servers' IP address (again, 192.168.3.5) From ?Port? choose ?993? From ?SSL? choose ?SSL/TLS? From ?Authentication? choose ?Autodetect? For ?Outgoing? the admin will let the ?SMTP? option For ?Server Hostname?, choose the same IP address (e.g. 192.168.3.5) For ?Port? choose ?25? For ?SSL? choose ?None? For ?Authentication? choose ?Normal Password? On the Username field, enter the full user name, e.g. ?user at domain.gr? Click on ?Re-test? Click on ?Create Account? Check on the ?I understand the Risks? checkbox? Click on ?Create Account? Click on ?Confirm Security Exception? Click on ?OK? Hope this helps and best regards, Spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From bangkokmaco at gmail.com Tue Sep 4 10:50:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Tue, 4 Sep 2012 14:50:40 +0700 Subject: [Dovecot] problems with ssl cert Message-ID: in conf.d/10-ssl.conf: ssl = yes ssl_cert = I upgraded dovecot from 1.1 to 2.1.8 pop3/imap works but i have a problem. i use pop3 protocol on outlook 2010 for my account. and i set keep my mails on the server for 5 days or until i delete them. These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. if i even delete a mail from outlook the server doesn't delete or old mails are not deleted. what can be the problem ? thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394.html Sent from the Dovecot mailing list archive at Nabble.com. From robert at schetterer.org Tue Sep 4 11:30:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 10:30:22 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST Message-ID: <5045BC1E.9040700@schetterer.org> Hi, perhaps somebody wants to this --snip Microsoft? Outlook? 2013 Preview Outlook 2013 Preview implements the IMAP LIST extension specified in [RFC6154] as the XLIST command. ---snip taken out of [MS-STANOIMAP].pdf which zip you may download here http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 11:34:59 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 09:34:59 +0100 (BST) Subject: [Dovecot] problems with ssl cert In-Reply-To: References: Message-ID: <1346747699.13635.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: "cc "maco" young" > To: dovecot at dovecot.org > Cc: > Sent: Tuesday, 4 September 2012, 10:50 > Subject: [Dovecot] problems with ssl cert > > in conf.d/10-ssl.conf: > > ssl = yes > ssl_cert = ssl_key = > when try to set up an account in thrunderbird, get /var/log/mail.log: > > imap-login: Disconnected (no auth attempts): rip=223.205.150.234, > lip=xx.xx.xx.xx, TLS: SSL_read() failed: error:14094416:SSL > routines:SSL3_READ_BYTES:sslv3 alert certificate unknown: SSL alert number > > openssl s_client -connect mail.myserver.com:993 - successful > > using the same cert in gning / chrome is successful, so think the cert is ok > Hi again, Do a fresh Thunderbird installation and get rid of any folders? related to thunderbird on either OS you have installed it on. s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From robert at schetterer.org Tue Sep 4 13:53:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 04 Sep 2012 12:53:33 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045BC1E.9040700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> Message-ID: <5045DDAD.3010705@schetterer.org> Am 04.09.2012 10:30, schrieb Robert Schetterer: > Hi, perhaps somebody wants to this > > --snip > Microsoft? Outlook? 2013 Preview > Outlook 2013 Preview implements the IMAP LIST extension specified in > [RFC6154] as the XLIST command. > ---snip > > taken out of [MS-STANOIMAP].pdf > > which zip you may download here > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > just for info, by small testing Outlook 2013 preview imap specialuse is working with dovecot at minimum for Trash and Sent i have set this mailbox Sent { special_use = \Sent auto=subscribe } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash auto=subscribe } examples are in /etc/dovecot/conf.d/15-mailboxes.conf couldnt test Archive/Drafts/Junk but they "look" Working i had my problems with some new gui functions *g the preview is only in english or spanish so dont know if it will work with other languages then english, lets hope so feel free to test yourself -- Best Regards MfG Robert Schetterer From stsiol at yahoo.co.uk Tue Sep 4 16:31:49 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Tue, 4 Sep 2012 14:31:49 +0100 (BST) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <1346738387.37361.YahooMailNeo@web132202.mail.ird.yahoo.com> <1346743882.18960.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <1346765509.32698.YahooMailNeo@web132201.mail.ird.yahoo.com> >________________________________ > From: "cc "maco" young" >To: Spyros Tsiolis >Sent: Tuesday, 4 September 2012, 16:00 >Subject: Re: [Dovecot] thunderbird not connecting > > >On Tue, Sep 4, 2012 at 2:31 PM, Spyros Tsiolis wrote: > >> . . . . ?%< . . . . . %< . . . . ?. . . . >? >what you showed is what I remember when I was last hacking in TB. ?now all options are gone. ?screenshot attached > > >think I know problem. ?got tired of TB - no options, no feedback. ?ok if everything works; otherwise sucks. > > >went to claws. ?they have actual messages, actual logs. ?had similar problem, but claws easily allowed me to override. ?see this:?http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2199 > > >know my cert chain ok because?http://www.sslshopper.com/ssl-checker.html checks chain ok > > >anyway, thanks for your help, and goodbye thunderbird > > > > I remember being stuck on this. I know what you are saying. Give it another go. Tinker with it. You'll get there. If you're fed up with TB, that's another thing. However, you _can_ get round this. I know I did. Just my 2c, Regards, spyros ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From dg at dguhl.org Tue Sep 4 18:20:08 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 4 Sep 2012 17:20:08 +0200 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <20120904152007.GA20273@PC211.ikt.de> On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: > Hi! [..] > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. http://wiki2.dovecot.org/Tools/Doveadm/Expunge HTH Dennis [..] From alessio at skye.it Tue Sep 4 18:40:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 04 Sep 2012 17:40:33 +0200 Subject: [Dovecot] doveadm command to block a user? In-Reply-To: <50446D34.5000702@um.es> References: <50446D34.5000702@um.es> Message-ID: <504620F1.5040801@skye.it> Il 03/09/2012 10:41, Angel L. Mateo ha scritto: > Hello, > > Is there any command to be able to lock imap access to a specific > user? With doveadm kick I could close his connections, but I want to > avoid future ones too and just for imap. > > Any way to do it? > I can do it by disable specific user at backend level, for example vpopmail can do it, but also with SQL can be simple. With specific query you can disable only IMAP, only POP and so on. In the password query you can insert something like this: WHERE username = '%u' AND active = '1' AND ((IF( %a=110, pop, 0 )) =1 OR (IF( %a=25, smtp , 0 )) =1 OR (IF(( %a = 143 ) AND '%l' = '127.0.0.1', webmail, 0 ) = 1 ) OR ( IF(( %a = 143 ) AND '%l' !='127.0.0.1', imap, 0 ) = 1)) Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From sven at svenhartge.de Tue Sep 4 18:40:17 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 17:40:17 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <20120904152007.GA20273@PC211.ikt.de> Message-ID: <0937s3tbapv8@mids.svenhartge.de> Dennis Guhl wrote: > On Mon, Sep 03, 2012 at 01:31:18PM +0200, Sven Hartge wrote: >> Hi! >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > http://wiki2.dovecot.org/Tools/Doveadm/Expunge Yes, I know about doveadm expunge. I am searching for a search_query which only matches messages with the flag \Deleted whose flag has been set 24h or longer ago. Doing something like doveadm expunge -A mailbox \* DELETED is not the solution, since it will expunge all deleted mails without looking at the time when they have been deleted. Using savedbefore does not work either, since a message may have been saved 2 years ago but only recently deleted. Gr??e, Sven. -- Sigmentation fault. Core dumped. From tss at iki.fi Tue Sep 4 19:11:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 19:11:35 +0300 Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: <4934p48so7v8@mids.svenhartge.de> References: <4934p48so7v8@mids.svenhartge.de> Message-ID: <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> On 3.9.2012, at 14.31, Sven Hartge wrote: > I want to use this to expunge all DELETED messages which have been in > that state for 24h or longer. Dovecot doesn't keep track of that information, so you can't. From sven at svenhartge.de Tue Sep 4 19:35:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Tue, 4 Sep 2012 18:35:51 +0200 Subject: [Dovecot] finding messages deleted before timespec References: <4934p48so7v8@mids.svenhartge.de> <90FF2FC6-A852-49C3-BCAF-A2053E762C26@iki.fi> Message-ID: <0937vh4hfsv8@mids.svenhartge.de> Timo Sirainen wrote: > On 3.9.2012, at 14.31, Sven Hartge wrote: >> I want to use this to expunge all DELETED messages which have been in >> that state for 24h or longer. > Dovecot doesn't keep track of that information, so you can't. I suspected this. Would be nice though. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Tue Sep 4 19:37:24 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:37:24 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? Message-ID: <50462E44.2090505@Media-Brokers.com> Almost every message I'm getting through this list is duplicated, down to the same exact message-ID... Anyone else seeing this? Charles From CMarcus at Media-Brokers.com Tue Sep 4 19:40:48 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 12:40:48 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50462F10.90906@Media-Brokers.com> On 2012-09-04 12:37 PM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Even this one was duplicated... From philip at turmel.org Tue Sep 4 19:58:01 2012 From: philip at turmel.org (Phil Turmel) Date: Tue, 04 Sep 2012 12:58:01 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <50463319.8060309@turmel.org> On 09/04/2012 12:40 PM, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Even this one was duplicated... Not here :-) Phil From stan at hardwarefreak.com Tue Sep 4 20:02:38 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:02:38 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <5046342E.4010808@hardwarefreak.com> On 9/3/2012 11:21 PM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx What does TB activity manager say? -- Stan From stan at hardwarefreak.com Tue Sep 4 20:07:05 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 04 Sep 2012 12:07:05 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462E44.2090505@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> Message-ID: <50463539.9090005@hardwarefreak.com> On 9/4/2012 11:37 AM, Charles Marcus wrote: > Almost every message I'm getting through this list is duplicated, down > to the same exact message-ID... > > Anyone else seeing this? Nope. Make any changes to Postfix or your script recently? -- Stan From florob at babelmonkeys.de Tue Sep 4 21:25:54 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 04 Sep 2012 20:25:54 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation Message-ID: <504647B2.5060804@babelmonkeys.de> Hello everyone and Timo in particular, about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present in Dovecot. I had always disliked this a bit, because it replicates a lot of code. This patch generalizes the HMAC function to take a hash_method struct as parameter, and changes existing code which uses the "old" HMAC function to use this new one. I'm not really sure this is actually a good idea, but I still felt I should provide the code in case you would want to merge it upstream. Attached is the patch as a hg export based on the revision of dovecot-2.2 current at the time of writing. Regards, Florian Zeitz -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID e2f682fab829c2ef23a050f884191f57e2fb5d60 # Parent 9bc2e718392ceaa327f14b50163232b629cd54d1 lib: Generalize hmac to be hash independent diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,64 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = hmac_ctx->ctx; + struct md5_context *ctxo = hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,12 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + ctx->hash = meth; + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,23 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +struct hmac_context { + void *ctx, *ctxo; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From rob0 at gmx.co.uk Tue Sep 4 22:16:50 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 4 Sep 2012 14:16:50 -0500 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50462F10.90906@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50462F10.90906@Media-Brokers.com> Message-ID: <20120904191649.GN3672@harrier.slackbuilds.org> On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: On Tue, Sep 04, 2012 at 12:40:48PM -0400, Charles Marcus wrote: > On 2012-09-04 12:37 PM, Charles Marcus > wrote: > >Almost every message I'm getting through this list is duplicated, > >down to the same exact message-ID... > > > >Anyone else seeing this? > > Even this one was duplicated... I think you're seeing double. Check to see if someone spiked your coffee. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From rplatel at tucows.com Tue Sep 4 23:01:45 2012 From: rplatel at tucows.com (Richard Platel) Date: Tue, 4 Sep 2012 16:01:45 -0400 Subject: [Dovecot] Custom auth process in dovecot 2 Message-ID: Hi, I'm trying to upgrade from dovecot-1.1.x to 2.1.7. We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. In dovecot 1.1 we could configure this with auth external { socket connect { master { path = /var/run/dovecot/auth.sock } } } as per http://wiki.dovecot.org/MainConfig I haven't been able to figure out how to do this in 2.1.7, is it possible? From tss at iki.fi Tue Sep 4 23:05:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 4 Sep 2012 23:05:55 +0300 Subject: [Dovecot] Custom auth process in dovecot 2 In-Reply-To: References: Message-ID: <99C78902-F077-4DB0-991C-39171320EF1E@iki.fi> On 4.9.2012, at 23.01, Richard Platel wrote: > I'm trying to upgrade from dovecot-1.1.x to 2.1.7. > > We have our own custom auth server process (because we want to do our own password validation and for other reasons) that listens on a UNIX domain socket and speaks the dovecot auth protocol. > > In dovecot 1.1 we could configure this with > > auth external { > socket connect { > master { > path = /var/run/dovecot/auth.sock > } > } > } > > as per http://wiki.dovecot.org/MainConfig > > I haven't been able to figure out how to do this in 2.1.7, is it possible? Possibility a) Disable the regular service auth, something like: service auth { unix_listener login/login { mode = 0 } } and then just create the /var/run/dovecot/login/login socket yourself. You may need/want to do this for other auth-related sockets too. Possibility b) Tell login processes to connect to your socket instead: service imap-login { executable = imap-login newlogin } From CMarcus at Media-Brokers.com Wed Sep 5 00:02:00 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 04 Sep 2012 17:02:00 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50463539.9090005@hardwarefreak.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> Message-ID: <50466C48.1020303@Media-Brokers.com> On 9/4/2012 1:07 PM, Stan Hoeppner wrote: > On 9/4/2012 11:37 AM, Charles Marcus wrote: >> Almost every message I'm getting through this list is duplicated, down >> to the same exact message-ID... >> >> Anyone else seeing this? > > Nope. Make any changes to Postfix or your script recently? Nope... but, your reply made me actually take a closer look... It is only happening for emails that are filtered to folders (like my lists)... And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... I'll disable filtering on one, and see if that solves the problem (I expect it will)... Sorry for the noise... -- Best regards, Charles From h.reindl at thelounge.net Wed Sep 5 00:57:43 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 04 Sep 2012 23:57:43 +0200 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50466C48.1020303@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> Message-ID: <50467957.2010602@thelounge.net> Am 04.09.2012 23:02, schrieb Charles Marcus: > On 9/4/2012 1:07 PM, Stan Hoeppner wrote: >> On 9/4/2012 11:37 AM, Charles Marcus wrote: >>> Almost every message I'm getting through this list is duplicated, down >>> to the same exact message-ID... >>> >>> Anyone else seeing this? >> >> Nope. Make any changes to Postfix or your script recently? > > Nope... but, your reply made me actually take a closer look... > > It is only happening for emails that are filtered to folders (like my lists)... > > And now I realize it must be because I've been keeping Thunderbird open at two separate locations (we just opened > another office about 5 minutes away, and I have an office/computer at both), and each has filtering enabled... > > I'll disable filtering on one, and see if that solves the problem (I expect it will)... better use SIEVE filters on the server for moving list-messages to folders instead the overhead of TB which is in fact copy (download/upload) followed by delete so you have no problems with concurrent clients including mobile devices -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From jtam.home at gmail.com Wed Sep 5 02:02:44 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 4 Sep 2012 16:02:44 -0700 (PDT) Subject: [Dovecot] finding messages deleted before timespec In-Reply-To: References: Message-ID: On Tue, 4 Sep 2012, Sven Hartge wrote: > I am searching for a search_query which only matches messages with the > flag \Deleted whose flag has been set 24h or longer ago. > > Doing something like > > doveadm expunge -A mailbox \* DELETED > > is not the solution, since it will expunge all deleted mails without > looking at the time when they have been deleted. > > Using savedbefore does not work either, since a message may have been > saved 2 years ago but only recently deleted. You could work out a cron job that does a "doveadm search ...", then diff it with one generated 24 hours ago, extract the common GID/UUIDs, then do a second pass to delete them. Not elegant, but it would probably work. (I just read the man page, and there doesn't seem to be an option to pipe the search like "doveadm search ... | doveadm expunge ...". You'll have to expunge one message at a time. Ugh.) Or maybe the advice "don't worry, be happy" applies here. I find that too many "mistake mitigation" features is counterproductive as users habitually relying on them, rather than being careful. Extending your grace period to a week will lessen the probability of this race condition. Joseph Tam From kgc at corp.sonic.net Wed Sep 5 02:06:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 16:06:37 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <57695CBD-70BA-4B15-85D1-4AABEAE92F29@iki.fi> Message-ID: <5046897D.1050502@corp.sonic.net> On 09/03/12 12:06, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. >> >> There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) > > Did you try stopping both and then starting them again? That clears up all the state they have. I stopped both directors last night and they were able to stay in sync after they were restarted. Could corruption of the in memory state lead to the connections being dropped? If this happens again I'll try to get a tcpdump and an strace so the bug can get squashed. -K From tss at iki.fi Wed Sep 5 03:58:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 03:58:19 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > passdb { > args = proxy=y nopassword=y > driver = static > } I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. From tss at iki.fi Wed Sep 5 04:23:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 04:23:39 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <6CB24D47-CD68-420B-A3E7-5C79DE1CD757@iki.fi> On 5.9.2012, at 3.58, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. Although the extra users should be freed from the memory after 15 minutes. Hmm. Once Dovecot supports moving existing connections from one backend server to another without the client noticing anything, the director could be simplified by using consistent hashing and when the number of backends changes, the director could start moving connections to their proper backends. During this move new connections would be handled by 1) if old backend = new backend just forward the connection there or 2) if they're different, request immediate move for that user's existing connections and wait for it to be finished before letting new connections finish. Or alternatively if the user isn't just being moved at that time, forward the connection to the old server and let it be part of the later move. The main difference here is that directors wouldn't need to keep any track of user -> backend associations. The moving period could still be a bit tricky to handle well, especially since the situation can change again while a previous move is still going on. From kgc at corp.sonic.net Wed Sep 5 07:06:51 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Tue, 04 Sep 2012 21:06:51 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <704F820B-C688-4E6E-9184-E9F12F457632@iki.fi> Message-ID: <5046CFDB.90809@corp.sonic.net> On 9/4/2012 5:58 PM, Timo Sirainen wrote: > On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >> passdb { >> args = proxy=y nopassword=y >> driver = static >> } > > I wonder if someone was doing a ton of logins for different usernames? This kind of setup where director doesn't verify the username can be attacked that way. It doesn't look like there was a higher than normal number of failed logins leading up to the connection issues. I'm going to write some more stats collection tools to track state on the directors and see what comes of it. Can the director proxy validate the username via a unix pw lookup but not check the password? -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From p at state-of-mind.de Wed Sep 5 09:14:10 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 08:14:10 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5045DDAD.3010705@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> Message-ID: <20120905061409.GC12086@state-of-mind.de> * Robert Schetterer : > Am 04.09.2012 10:30, schrieb Robert Schetterer: > > Hi, perhaps somebody wants to this > > > > --snip > > Microsoft? Outlook? 2013 Preview > > Outlook 2013 Preview implements the IMAP LIST extension specified in > > [RFC6154] as the XLIST command. > > ---snip > > > > taken out of [MS-STANOIMAP].pdf > > > > which zip you may download here > > > > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 > > > > just for info, by small testing > Outlook 2013 preview imap specialuse is working with dovecot > at minimum for Trash and Sent > > i have set this > > mailbox Sent { > special_use = \Sent > auto=subscribe > } > mailbox "Sent Messages" { > special_use = \Sent > } Why did you set the \Sent folder twice? Setting it once should suffice. > the preview is only in english or spanish > so dont know if it will work with other languages then english, lets hope so Try this: mailbox "Gesendete Objekte" { special_use = \Sent auto=subscribe } AFAIK the English version should automagically map itself to that folder. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From duihi77 at gmail.com Wed Sep 5 09:45:09 2012 From: duihi77 at gmail.com (Duane Hill) Date: Wed, 5 Sep 2012 06:45:09 +0000 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <723005990.20120905064509@gmail.com> On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > * Robert Schetterer : >> Am 04.09.2012 10:30, schrieb Robert Schetterer: >> > Hi, perhaps somebody wants to this >> > >> > --snip >> > Microsoft? Outlook? 2013 Preview >> > Outlook 2013 Preview implements the IMAP LIST extension specified in >> > [RFC6154] as the XLIST command. >> > ---snip >> > >> > taken out of [MS-STANOIMAP].pdf >> > >> > which zip you may download here >> > >> > http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >> > >> >> just for info, by small testing >> Outlook 2013 preview imap specialuse is working with dovecot >> at minimum for Trash and Sent >> >> i have set this >> >> mailbox Sent { >> special_use = \Sent >> auto=subscribe >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } > Why did you set the \Sent folder twice? Setting it once should suffice. FYI: Default config shows it listed twice in: conf.d/15-mailboxes.conf ... # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } ... -- If at first you don't succeed... ...so much for skydiving. From robert at schetterer.org Wed Sep 5 10:18:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:18:14 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <20120905061409.GC12086@state-of-mind.de> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> Message-ID: <5046FCB6.7080700@schetterer.org> Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > Try this: > > mailbox "Gesendete Objekte" { > special_use = \Sent > auto=subscribe > } i will do when Outlook 2013 in german got released > > AFAIK the English version should automagically map itself to that folder. agree ,it should > > p at rick thunderbirds status about xlist etc can be seen here https://bugzilla.mozilla.org/show_bug.cgi?id=558659 looks like , its on the road, from pure tec side no idea if and when it will go released -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 5 10:28:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 05 Sep 2012 09:28:53 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <723005990.20120905064509@gmail.com> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <723005990.20120905064509@gmail.com> Message-ID: <5046FF35.4070504@schetterer.org> Am 05.09.2012 08:45, schrieb Duane Hill: > On Wednesday, September 05, 2012 at 06:14:10 UTC, p at state-of-mind.de confabulated: > >> * Robert Schetterer : >>> Am 04.09.2012 10:30, schrieb Robert Schetterer: >>>> Hi, perhaps somebody wants to this >>>> >>>> --snip >>>> Microsoft? Outlook? 2013 Preview >>>> Outlook 2013 Preview implements the IMAP LIST extension specified in >>>> [RFC6154] as the XLIST command. >>>> ---snip >>>> >>>> taken out of [MS-STANOIMAP].pdf >>>> >>>> which zip you may download here >>>> >>>> http://msdn.microsoft.com/en-us/library/ee157124%28v=exchg.80%29 >>>> >>> >>> just for info, by small testing >>> Outlook 2013 preview imap specialuse is working with dovecot >>> at minimum for Trash and Sent >>> >>> i have set this >>> >>> mailbox Sent { >>> special_use = \Sent >>> auto=subscribe >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } > >> Why did you set the \Sent folder twice? Setting it once should suffice. > > FYI: Default config shows it listed twice in: > > conf.d/15-mailboxes.conf > ... > # For \Sent mailboxes there are two widely used names. We'll mark both of > # them as \Sent. User typically deletes one of them if duplicates are created. > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > ... > "Sent Messages" was/is used in some clients for Sent, guess its also in the example, looks like the new client versions with xlist etc changed this sometimes,too ( going directly Sent , but i didnt investigated hardly in this, test yourself), so it may not longer needed in the future, meanwhile it does not hurt somenone, since yet But i agree feature need more real world examples and explains in some wiki related to div mail clients in their different versions guess this is on the road -- Best Regards MfG Robert Schetterer From dafan.zhai at securepoint.de Wed Sep 5 11:53:37 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Wed, 05 Sep 2012 10:53:37 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder Message-ID: <50471311.2050002@securepoint.de> Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, &metadata); but Valgrind finds a memory leak when this function is called: ---------------------------------------------------snip-------------------------------------------------------------- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304== by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---------------------------------------------------snip-------------------------------------------------------------- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---------------------------------------------------snip-------------------------------------------------------------- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---------------------------------------------------snip-------------------------------------------------------------- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1 branch fixing memory leaks: http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b It would be nice if you could give me some hints how I can address this issue? Thanks in advance! Greetings, Dafan From lgb at lgb.hu Wed Sep 5 13:20:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 5 Sep 2012 12:20:26 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? Message-ID: <20120905102026.GB23439@vega.lgb.hu> Dear All, I have a bit complex question about ldap based user/passdb with some twists caused by the LDAP schema I have to use. It's complex for me at least, since it's my first time to try to setup any dovecot install which is more than a "trivial" thing eg for some unix system users. What I'd like to do: Users are stored in LDAP with the following schema: storageMailUid: contains user's uid in x at z format, and it CAN be different than mail address. This value is used to login (pop3/imap) and to get the maildir path (well, home) in the form of /mailstorage/z/x/ regardless of the value of the "mail" attribute. If @z part is missing (I mean the user supplied login, not the LDAP!), some default value (example.com) should be treated (I guess that's easy: auth_default_realm may be enough, isn't it). It is NOT possible to login via pop3/imap with mail, only with storageMailUid! This storageMailUid can specify a string which is not even an existing mail address, of course. mail: contains user's mail address, or even mail addresses (there can be more mail addresses - more mail ldap attributes - for a single ldap entry). This attribute cannot be used for pop3/imap login, neither it counts to resolve the maildir path, it only counts when a mail is received: the location of the user's maildir must be got from storageMailUid. storageMailQuota: it contains the quota value (in bytes) for the given user, which should be enforced on receiving mail, or by using IMAP (not counting the Trash folder which is a fixed size for all users and it must be handled outside of this). All user maildirs have the same fixed unix UID/GID, which is not stored in LDAP, but must be configured statically. "Home directory" of the user is also not stored in LDAP, it must be got from storageMailUid LDAP parameter. Mails are received via LMTP only. I'd like to use prefetchdb to minimalize the amount of LDAP lookups (however I can live without that). I am unsure if auth_bind is OK, or it should not be used, currently I'd like to play with auth_bind, since it worked before on other servers well. I'm totally lost with the user_attrs/pass_attrs to create this kind of configuration. In case of static userdb, it was easy to set up: passdb { args = /etc/dovecot/dovecot-ldap-passdb.conf driver = ldap } userdb { args = uid=vmail gid=vmail home=/mailstorage/%Ld/%Ln driver = static } With this, pop3/imap worked (quota was not of the scope yet, though), however lmtp is not ("passdb doesn't support lookups, can't verify user's existence" which is odd for me, as userdb does not support it in my opinion). Please, give some suggestion how to set up userdb and passdb to support this configuration which works with the described scenario. I should also set iterate_filter and iterate_attrs I guess, so some doveadm commands can work then (-A stuffs, I guess). I am also not sure if auth service should be used or not (I mean extra configuration related): since I want LMTP not LDA, I guessed I don't need it, but I am not sure. I am also confused, because on receiving a mail (via LMTP) different kind of LDAP lookup is needed: then mail must be searched, but it's storageMailUid based lookup in case of pop3 or imap login ... Is it possible to give different userdb/passdb for lmtp and pop3/imap? In theory it's even possible to have x at example.com as mail and y at example.com as storageMailUid for one user, and the opposite for the another, so lookups cannot be done together for mail and storageMailUid. The LDAP schema/rest of the system works this way, not an option to change. Any help is greatly welcomed. Thanks a lot in advance, G?bor From bangkokmaco at gmail.com Wed Sep 5 13:23:25 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 17:23:25 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <5046342E.4010808@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner wrote: > On 9/3/2012 11:21 PM, cc young wrote: > > cannot get TB to recognize either pop3/s or imap/s server > > > > can connect just fine with: > > > > openssl s_client -connect ms1.myserver.net:993 > > . login ... > > > > but trying with TB /var/log/mail.log gets: > > > > dovecot: pop3-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > dovecot: imap-login: Aborted login (no auth attempts): > rip=223.205.150.234, > > lip=xxx.xx.xx.xx > > What does TB activity manager say? > > Activity Manager is blank as an update, went to Claws email client. it had enough feedback and gave me enough ways to tweak - got everything going. went back to TB. recognized server, but no username/password. perhaps it's not letting the full email - which is the user name - through. but who knows? no feedback. From CMarcus at Media-Brokers.com Wed Sep 5 13:58:02 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 06:58:02 -0400 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <50467957.2010602@thelounge.net> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> Message-ID: <5047303A.5060609@Media-Brokers.com> On 2012-09-04 5:57 PM, Reindl Harald wrote: > better use SIEVE filters on the server I know, it is on my ToDo list... we only just recently migrated this server to Dovecot, and I've had my plate full with other issues, which are now mostly resolved, so I'm about ready to circle back and finish up (installing SOGo, enabling sieve, etc), but I need to spin up a new Gentoo VM first, because I want to get the current server moved over to our new VMWare environment. -- Best regards, Charles From keijser at stone-it.com Wed Sep 5 14:00:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 13:00:04 +0200 Subject: [Dovecot] managesieve problem Message-ID: <1346842804.29847.13.camel@localhost> Hi, I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) and pigeonhole (2.0.9). All works well except sieve. I've already posted on the roundcube forum (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was told I should ask here since it's more likely a dovecot issue rather than roundcube. The problem is that if I want to manage sieve filters from roundcube, I get this error in maillog: Sep 5 12:55:39 emperor dovecot: managesieve-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=1575, secured Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: stat((null)) failed: Bad address Sep 5 12:55:39 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage The managesieve daemon is listening though: emperor ~ # netstat -tlpne | grep dovecot tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 0 1275042 362/dovecot tcp 0 0 0.0.0.0:2000 0.0.0.0:* LISTEN 0 1275037 362/dovecot tcp 0 0 0.0.0.0:4190 0.0.0.0:* LISTEN 0 1275035 362/dovecot tcp 0 0 0.0.0.0:993 0.0.0.0:* LISTEN 0 1275044 362/dovecot tcp 0 0 :::143 :::* LISTEN 0 1275043 362/dovecot tcp 0 0 :::2000 :::* LISTEN 0 1275038 362/dovecot tcp 0 0 :::4190 :::* LISTEN 0 1275036 362/dovecot tcp 0 0 :::993 :::* LISTEN 0 1275045 362/dovecot emperor ~ # telnet localhost 4190 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus" "NOTIFY" "mailto" "SASL" "PLAIN" "STARTTLS" "VERSION" "1.0" OK "Dovecot ready." Before I paste any configuration files, I wanted to ask first if this is a common/known issue. Else, please let me know what additional information I should provide. Thanks in advance! kind regards, L?on From CMarcus at Media-Brokers.com Wed Sep 5 14:02:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 07:02:32 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> Message-ID: <50473148.8050905@Media-Brokers.com> On 2012-09-05 6:23 AM, cc maco young wrote: > On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >> What does TB activity manager say? > Activity Manager is blank > > as an update, went to Claws email client. it had enough feedback and gave > me enough ways to tweak - got everything going. > > went back to TB. recognized server, but no username/password. perhaps > it's not letting the full email - which is the user name - through. but > who knows? no feedback. You're looking at the wrong end. You need to be looking at your SERVER logs, not the Clients. -- Best regards, Charles From stephan at rename-it.nl Wed Sep 5 14:16:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 13:16:19 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346842804.29847.13.camel@localhost> References: <1346842804.29847.13.camel@localhost> Message-ID: <50473483.9020702@rename-it.nl> Op 9/5/2012 1:00 PM, L?on Keijser schreef: > Hi, > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > and pigeonhole (2.0.9). All works well except sieve. I've already posted > on the roundcube forum > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > told I should ask here since it's more likely a dovecot issue rather > than roundcube. [...] > Before I paste any configuration files, I wanted to ask first if this is > a common/known issue. Else, please let me know what additional > information I should provide. The issue does look familiar and most likely it is already solved. But, the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. Upgrade first. Regards, Stephan. From bangkokmaco at gmail.com Wed Sep 5 14:18:40 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 5 Sep 2012 18:18:40 +0700 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: On Wed, Sep 5, 2012 at 6:02 PM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: > >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppner >> >wrote: >> >>> What does TB activity manager say? >>> >> > Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. >> > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. > > yes, you are right. will get back. From amateo at um.es Wed Sep 5 14:40:54 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 05 Sep 2012 13:40:54 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) Message-ID: <50473A46.1080807@um.es> Hello, We have a dovecot servers farm with storage in NFS and indexes in local files. Under heavy load conditions we a log of messages in the form: Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. PS: I have attached my dovecot -n output -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_privileged_group = mail mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = maildir:User quota quota_rule = *:storage=20G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es service anvil { client_limit = 2051 } service auth { client_limit = 3072 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From tss at iki.fi Wed Sep 5 14:50:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 5 Sep 2012 14:50:38 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50473A46.1080807@um.es> References: <50473A46.1080807@um.es> Message-ID: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> On 5.9.2012, at 14.40, Angel L. Mateo wrote: > Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) > > We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). > > So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). From zucca at systemschmiede.com Wed Sep 5 14:59:34 2012 From: zucca at systemschmiede.com (Sascha Zucca) Date: Wed, 05 Sep 2012 13:59:34 +0200 Subject: [Dovecot] strange Folder in subscription list (was: Imap Ghost Folder) Message-ID: <50473EA6.3020203@systemschmiede.com> Hi list, here's the complete Problem for easy reading again. Can someone reproduce this? I have a strange "ghostfolder" when using shares/%%n/ as prefix for shared namespace. If i swith only this setting to shares/%%u/ everything is just as wanted, only then the prefix is very long, so i would prefer the name. dovecot --version 2.1.7 dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-14-pve i686 Debian 6.0.5 simfs auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%%d/%%n/Maildir mail_plugins = acl mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/shared/%%u prefix = shares/%%n/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } ssl_cert = took seconds) In-Reply-To: <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> Message-ID: <50474093.3010306@um.es> El 05/09/12 13:50, Timo Sirainen escribi?: > On 5.9.2012, at 14.40, Angel L. Mateo wrote: > >> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >> >> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >> >> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. > > That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). > Ok, I'll try this. But... is there any way to know how a readdir would be done? From keijser at stone-it.com Wed Sep 5 15:49:39 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 14:49:39 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <50473483.9020702@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> Message-ID: <1346849379.4102.3.camel@localhost> Hi Stephan, On Wed, 2012-09-05 at 13:16 +0200, Stephan Bosch wrote: > Op 9/5/2012 1:00 PM, L?on Keijser schreef: > > Hi, > > > > I'm using roundcube in combination with postfix (2.6.6), dovecot (2.0.9) > > and pigeonhole (2.0.9). All works well except sieve. I've already posted > > on the roundcube forum > > (http://www.roundcubeforum.net/index.php/topic,9690.0.html ) but was > > told I should ask here since it's more likely a dovecot issue rather > > than roundcube. > [...] > > Before I paste any configuration files, I wanted to ask first if this is > > a common/known issue. Else, please let me know what additional > > information I should provide. > > The issue does look familiar and most likely it is already solved. But, > the latest release of Dovecot v2.0 is v2.0.21, making your version ancient. > > Upgrade first. Upgraded to 2.0.21. Now I get this error: Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory for substitition in storage root directory (sieve_dir=~/sieve) Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: Failed to open Sieve storage. regards, L?on From keijser at stone-it.com Wed Sep 5 16:26:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Wed, 05 Sep 2012 15:26:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346849379.4102.3.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> Message-ID: <1346851564.4102.7.camel@localhost> On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: > Upgraded to 2.0.21. Now I get this error: > > > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: > sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory > for substitition in storage root directory (sieve_dir=~/sieve) > Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: > Failed to open Sieve storage. I should probably mention that I have virtual domains/users, all located in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a real user on the box. regards, L?on From stephan at rename-it.nl Wed Sep 5 16:29:46 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 05 Sep 2012 15:29:46 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <1346851564.4102.7.camel@localhost> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> Message-ID: <504753CA.3060609@rename-it.nl> Op 9/5/2012 3:26 PM, L?on Keijser schreef: > On Wed, 2012-09-05 at 14:49 +0200, L?on Keijser wrote: >> Upgraded to 2.0.21. Now I get this error: >> >> >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Error: >> sieve-storage: userdb(leon at EXAMPLE.COM) didn't return a home directory >> for substitition in storage root directory (sieve_dir=~/sieve) >> Sep 5 14:34:29 emperor dovecot: managesieve(leon at EXAMPLE.COM): Fatal: >> Failed to open Sieve storage. > > I should probably mention that I have virtual domains/users, all located > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > real user on the box. This should provide all the info you need on that: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Stephan. From p at state-of-mind.de Wed Sep 5 19:22:21 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 5 Sep 2012 18:22:21 +0200 Subject: [Dovecot] Outlook 2013 imap specialuse RFC6154 XLIST In-Reply-To: <5046FCB6.7080700@schetterer.org> References: <5045BC1E.9040700@schetterer.org> <5045DDAD.3010705@schetterer.org> <20120905061409.GC12086@state-of-mind.de> <5046FCB6.7080700@schetterer.org> Message-ID: <20120905162219.GB24756@state-of-mind.de> * Robert Schetterer : > Am 05.09.2012 08:14, schrieb Patrick Ben Koetter: > > Try this: > > > > mailbox "Gesendete Objekte" { > > special_use = \Sent > > auto=subscribe > > } > > i will do when Outlook 2013 in german got released > > > > AFAIK the English version should automagically map itself to that folder. > > agree ,it should > > > > > p at rick > > thunderbirds status about xlist etc can be seen here > > https://bugzilla.mozilla.org/show_bug.cgi?id=558659 > > looks like , its on the road, from pure tec side > no idea if and when it will go released It is implemented. We had to adapt a few extra functions to deal with 64 bit stuff. Currently it is not being pushed further because of the uncertainty of TBs future. Once it will become clear how new features will be released we will spend the rest of money and time to ship the feature. For now I will not spend a single more Euro. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From CMarcus at Media-Brokers.com Thu Sep 6 00:03:27 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 05 Sep 2012 17:03:27 -0400 Subject: [Dovecot] mail_location / mail_home In-Reply-To: <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> References: <38719C76-8672-4F85-B4EA-8802CF85BE63@4lin.net> <87817242-5739-4FA8-9A52-400E4E7DBA2B@4lin.net> Message-ID: <5047BE1F.2000103@Media-Brokers.com> On 2012-08-31 4:15 AM, Denny Schierz wrote: > I think, it doesn't make any difference, if I have "~" or a fixed path > :-/ On the productive system (also 2.1.9) I get messages like: [...] > dovecot.lda-dupes/tmp) failed: Not a directory [...] So, I need a way, > to move ~11.000 Maildirs .... any suggestions? mmv is a godsend... http://linux.dsplabs.com.au/mmv-copy-append-link-move-multiple-files-under-linux-shell-bash-by-wildcard-patterns-p5/ -- Best regards, Charles From ben at indietorrent.org Thu Sep 6 01:20:57 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 05 Sep 2012 18:20:57 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot Message-ID: <5047D049.8080004@indietorrent.org> Hello, I am attempting to use the Antispam plug-in for Dovecot and am having trouble with the implementation. My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. Mail that SpamAssassin tags as SPAM is moved into the affected user's "Junk" folder (/var/vmail/example.com/user/Maildir/.Junk). I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam . After restarting Dovecot, my IMAP client (Thunderbird v15) reports the following whenever I attempt to move mail from Inbox to "Junk" directory: "[CANNOT] antispam plugin not configured" That exact phrase appears only once in a Google search ( http://dovecot-antispam.sourcearchive.com/documentation/1.1plus-p20090218.git.g28075fa/mailtrain_8c-source.html ): if (!hamaddr || !spamaddr) { mail_storage_set_error(t->box->storage, ME(NOTPOSSIBLE) "antispam plugin not configured"); return -1; } Based on the surrounding source code, which I realize may be old, it seems that Ham and Spam addresses are required, or similar. I am using the following configuration options: /etc/dovecot/dovecot.conf -------------------- ## IMAP specific settings protocol imap { mail_executable = /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_plugins = quota imap_quota antispam } # [...] plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_spam_pattern_ignorecase = spam;junk } -------------------- For what it's worth, the logging directives seem not to have any effect (and perhaps the rest of the directives do not, either, hence the problem I'm having). I tried changing the above directives to their older directive names (I also changed the order slightly, which may be irrelevant): -------------------- plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_spam_pattern_ignorecase = spam;junk } -------------------- This changed the message in my IMAP client, when attempting to move a message from my Inbox to the Junk folder, to: "[SERVERBUG] failed to send mail" Does this indicate a problem with the "/usr/bin/sa-learn-pipe.sh" script that is provided on the Wiki (cited above)? Also, it was not clear from the documentation ( http://johannes.sipsolutions.net/files/antispam.html ) whether or not the Pipe back-end requires the X-DSPAM-Signature header, and if so, to what value it should be set when using SpamAssassin. Thanks for any pointers, -Ben From jtam.home at gmail.com Thu Sep 6 01:22:31 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 5 Sep 2012 15:22:31 -0700 (PDT) Subject: [Dovecot] thunderbird not connecting In-Reply-To: References: Message-ID: On Wed, 5 Sep 2012, Charles Marcus wrote > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. Actually, I think he did supply that and it showed imap and pop3 not getting any auth information. > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, lip=xxx.xx.xx.xx There are other log options to verbosely log auth and ssl information, and that might help. I frequently see the above log messages when the client and the server get their security protocol (SSL/TLS) and ports crossed up (i.e. trying IMAP/SSL on port 143 or trying IMAP/TLS on port 993). Try fiddling with TB's setting here and see if anything shakes loose. You can also use openssl and simulate the server end and maybe that will shed light on the situation: (You'll have to read the man page yourself to find all the options) openssl s_server -accept 993 ... Joseph Tam From kgc at corp.sonic.net Thu Sep 6 03:17:46 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 05 Sep 2012 17:17:46 -0700 Subject: [Dovecot] dovecot stats: useful data to gather In-Reply-To: <20120603001049.GA10970@daniel.localdomain> References: <20120601205839.GG2176@state-of-mind.de> <20120603001049.GA10970@daniel.localdomain> Message-ID: <5047EBAA.8090500@corp.sonic.net> On 06/02/12 17:10, Daniel Parthey wrote: > Patrick Ben Koetter wrote: >> following our discussion on dovecot stats at the LinuxTag 2012 my team and I >> sat down and put together a list of stat items we think to be useful in daily >> dovecot usage. >> >> Besides pulling together all the data we also think it would be useful to have >> an SNMP interface to access the stats. Our offer to create and contribute a >> standalone web interface for dovecot stats stands. > > This should be done via SNMP subagent, but how could you differentiate > different dovecot instances on the same machine, different snmp ports > for the subagent, or different snmp trees? I'd suggest some additional performance metrics like min/max/avg time to authenicate, establish a proxy session and perhaps include auth failure causes counters as well. I personally wouldn't want to see this implemented as an SNMP subagent but so long as the stats would be available off a local socket directly I think everyone would be happy. -K From stan at hardwarefreak.com Thu Sep 6 05:36:02 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 05 Sep 2012 21:36:02 -0500 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50473148.8050905@Media-Brokers.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> Message-ID: <50480C12.5040500@hardwarefreak.com> On 9/5/2012 6:02 AM, Charles Marcus wrote: > On 2012-09-05 6:23 AM, cc maco young wrote: >> On Wed, Sep 5, 2012 at 12:02 AM, Stan >> Hoeppnerwrote: >>> What does TB activity manager say? > >> Activity Manager is blank >> >> as an update, went to Claws email client. it had enough feedback and >> gave >> me enough ways to tweak - got everything going. >> >> went back to TB. recognized server, but no username/password. perhaps >> it's not letting the full email - which is the user name - through. but >> who knows? no feedback. > > You're looking at the wrong end. > > You need to be looking at your SERVER logs, not the Clients. He'd already checked the server logs, posted some of them, and found no useful information. Checking the client log is the next logical step, whether it turns out to contain useful information or not. In a client/server application, instructing someone to only check half the logs is bad advice Charles. Useful troubleshooting information can be found in either, or both, depending on the circumstances and operation that's failing. -- Stan From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 6 05:50:00 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 6 Sep 2012 04:50:00 +0200 Subject: [Dovecot] Length of attachment filenames In-Reply-To: References: Message-ID: <20120906025000.GA11887@daniel.localdomain> Fuhrmann, Hauke wrote: > I have a beginners question to Dovecot: does it have a restriction on > the supported length and character encodings of filenames of > attachments? > > Can filenames be arbitrary large? The maximum filename length depends largely on the file system: http://en.wikipedia.org/wiki/Comparison_of_file_systems Regards Daniel -- https://plus.google.com/103021802792276734820 From keijser at stone-it.com Thu Sep 6 09:25:04 2012 From: keijser at stone-it.com (=?ISO-8859-1?Q?L=E9on?= Keijser) Date: Thu, 06 Sep 2012 08:25:04 +0200 Subject: [Dovecot] managesieve problem In-Reply-To: <504753CA.3060609@rename-it.nl> References: <1346842804.29847.13.camel@localhost> <50473483.9020702@rename-it.nl> <1346849379.4102.3.camel@localhost> <1346851564.4102.7.camel@localhost> <504753CA.3060609@rename-it.nl> Message-ID: <1346912704.32331.2.camel@localhost> On Wed, 2012-09-05 at 15:29 +0200, Stephan Bosch wrote: > > I should probably mention that I have virtual domains/users, all located > > in /home/vmail/EXAMPLE.COM/. Usually the virtual user is not a > > real user on the box. > > This should provide all the info you need on that: > > http://wiki2.dovecot.org/VirtualUsers/Home Works like a charm now, thanks! :) regards, L?on From lists at wildgooses.com Thu Sep 6 09:27:47 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 07:27:47 +0100 Subject: [Dovecot] Anyone else seeing lots of random duplicate messages??? In-Reply-To: <5047303A.5060609@Media-Brokers.com> References: <50462E44.2090505@Media-Brokers.com> <50463539.9090005@hardwarefreak.com> <50466C48.1020303@Media-Brokers.com> <50467957.2010602@thelounge.net> <5047303A.5060609@Media-Brokers.com> Message-ID: <50484263.5040208@wildgooses.com> On 05/09/2012 11:58, Charles Marcus wrote: > I know, it is on my ToDo list... we only just recently migrated this > server to Dovecot, and I've had my plate full with other issues, which > are now mostly resolved, so I'm about ready to circle back and finish > up (installing SOGo, enabling sieve, etc), I have recently noticed owncloud (even has an ebuild for it). Have you re-evaluated roundcube+owncloud vs SOGo for a dav calender/contacts solution? Ed From CMarcus at Media-Brokers.com Thu Sep 6 13:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:10:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5047D049.8080004@indietorrent.org> References: <5047D049.8080004@indietorrent.org> Message-ID: <50487684.8060404@Media-Brokers.com> On 2012-09-05 6:20 PM, Ben Johnson wrote: > My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu 10.04. 1.2.9 is really old... you really need to upgrade to a recent/stable version. > I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam That is for version 2.x (note the wiki2)... I recommend going ahead and upgrading to the latest 2.1.9 and starting over. -- Best regards, Charles From CMarcus at Media-Brokers.com Thu Sep 6 13:17:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 06 Sep 2012 06:17:21 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <50480C12.5040500@hardwarefreak.com> References: <1346732514245-37389.post@n4.nabble.com> <5046342E.4010808@hardwarefreak.com> <50473148.8050905@Media-Brokers.com> <50480C12.5040500@hardwarefreak.com> Message-ID: <50487831.3060103@Media-Brokers.com> On 2012-09-05 10:36 PM, Stan Hoeppner wrote: > He'd already checked the server logs, posted some of them, and found > no useful information. Checking the client log is the next logical > step, whether it turns out to contain useful information or not. In a > client/server application, instructing someone to only check half the > logs is bad advice Charles. Useful troubleshooting information can be > found in either, or both, depending on the circumstances and operation > that's failing. That's what I get for replying in the middle of a thread without reading the whole thing... sorry... Been really busy at $dayjob, and haven't had time to read the lists I'm on for a couple of weeks now. And I didn't advise him to *only* check half the logs, I thought he was *starting out* with the client logs. Of course you're correct that once he'd determined that no auth attempts were happening, checking the client side is the next logical step. But then someone else suggested that it was just because Thunderbird was crapware, which is just plain stupid. If Thunderbird isn't AUTH'ing, it is a config (thus, user) error. -- Best regards, Charles From lgb at lgb.hu Thu Sep 6 14:06:34 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Thu, 6 Sep 2012 13:06:34 +0200 Subject: [Dovecot] different userdb and/or passdb for lmtp and pop3/imap? In-Reply-To: <20120905102026.GB23439@vega.lgb.hu> References: <20120905102026.GB23439@vega.lgb.hu> Message-ID: <20120906110634.GA12495@vega.lgb.hu> Hi Again, On Wed, Sep 05, 2012 at 12:20:26PM +0200, G?bor L?n?rt wrote: [...] > I am also confused, because on receiving a mail (via LMTP) different > kind of LDAP lookup is needed: then mail must be searched, but it's > storageMailUid based lookup in case of pop3 or imap login ... Is it > possible to give different userdb/passdb for lmtp and pop3/imap? Ok, after some "serious" google usage, I found a message in the mail list archive, that it helps to put userdb/passdb section inside the protocol specific part of the configuration. So I have something now like this: protocol pop3 { [...] } protocol imap { [...] } protocol lmtp { [...] passdb { [...] } userdb { [...] } } passdb { [...] } userdb { [...] } I have the idea, that in this way, lmtp should use dbs specified in the lmtp specific protocol settings, any other stuffs will use the settings at the "root level" of the configuration (this also includes iteration specific filter). However eg if I try to deliver a mail through lmtp in this way to a non-existing mail user, I can see in the logs, that lmtp after trying the specific dbs, it will also try the "global" ones, which is not good for me, as it can cause mis-deliveries instead of rejecting (as I have "some at thing" formatted %u for both of uid and mail but often they are not the same for the same user). I would be able to put db specifications into pop3 and imap (so not "global" configuration for them) but I guess in this case other services may (?) fail, like doveadm stuffs with CLI swtich -A (or am I wrong here?). Also it's not as nice, since then I must duplicate the same db specifications in both of imap and pop3 protocol specification parts of the configuration even they are the very same. Is there any idea to fix this little problem and/or a suggestion to solve my problem in a more elegant way than I tried to do? Thanks a lot in advance. - G?bor From dafan.zhai at securepoint.de Thu Sep 6 19:19:27 2012 From: dafan.zhai at securepoint.de (Dafan Zhai) Date: Thu, 06 Sep 2012 18:19:27 +0200 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <50471311.2050002@securepoint.de> References: <50471311.2050002@securepoint.de> Message-ID: <5048CD0F.9030809@securepoint.de> Hi I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Dafan On 09/05/2012 10:53 AM, Dafan Zhai wrote: > Hi everyone, > > I am writing a dovecot statistic plugin, which calls the > 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as > the 2nd parameter. > > enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; > struct mailbox_metadata metadata; > mailbox_get_metadata(mailbox, metadata_items, &metadata); > > but Valgrind finds a memory leak when this function is called: > ---------------------------------------------------snip-------------------------------------------------------------- > > ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 > of 76 > ==10304== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) > ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) > ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) > ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) > ==10304== by 0x4067102: maildir_mailbox_get_metadata > (maildir-storage.c:486) > ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) > ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) > ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) > ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) > ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) > ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) > ---------------------------------------------------snip-------------------------------------------------------------- > > > I have looked into the source code, and found that from the > 'mail_search_build_init' function call in > lib-storage/index/index-status.c:200 a pool is created, but the pool is > not freed in the mailbox_search_deinit function call in > lib-storage/index/index-status.c:218. > This may be the reason of the memory leak. But I do not know how to free > the pool. > > It seems to me that doveadm is having the same memory leak because it is > using 'mail_search_build_init' too. > ---------------------------------------------------snip-------------------------------------------------------------- > > # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize > INBOX > ==10457== Memcheck, a memory error detector > ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. > ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info > ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX > ==10457== > INBOX vsize=41643319 > ==10457== > ==10457== HEAP SUMMARY: > ==10457== in use at exit: 4,356 bytes in 3 blocks > ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated > ==10457== > ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 > ==10457== at 0x40222A4: calloc (in > /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) > ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x41AF16C: pool_alloconly_create (in > /usr/lib/dovecot/libdovecot.so.0.0.0) > ==10457== by 0x40DC07D: mail_search_build_init (in > /usr/lib/dovecot/libdovecot-storage.so.0.0.0) > ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in > /usr/bin/doveadm) > ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) > ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) > ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) > ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) > ==10457== by 0x805B863: main (in /usr/bin/doveadm) > ==10457== > ==10457== LEAK SUMMARY: > ==10457== definitely lost: 4,096 bytes in 1 blocks > ==10457== indirectly lost: 0 bytes in 0 blocks > ==10457== possibly lost: 0 bytes in 0 blocks > ==10457== still reachable: 260 bytes in 2 blocks > ==10457== suppressed: 0 bytes in 0 blocks > ==10457== Reachable blocks (those to which a pointer was found) are not > shown. > ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes > ==10457== > ==10457== For counts of detected and suppressed errors, rerun with: -v > ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) > ---------------------------------------------------snip-------------------------------------------------------------- > > > Dovecot version: 2.1.9 > OS: Linux 3.0.30-dist i686 > maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX > > I have applied the following patches from the dovecot 2.1 branch fixing > memory leaks: > http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 > http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b > > > It would be nice if you could give me some hints how I can address this > issue? > > Thanks in advance! > > Greetings, > Dafan -- Follow us on Facebook: Follow us on Twitter: --------------------------------------------------------------------- Securepoint GmbH | Entwicklung Dafan Zhai Salzstr. 1 D-21335 Lueneburg http://www.securepoint.de Tele: ++49 4131 2401-0 Fax: ++49 4131 2401-50 Lueneburg HRB 1776 --------------------------------------------------------------------- CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. GEHEIMHALTUNGSPFLICHT : Dieses E-Mail und alle damit verbundenen Anlagen sind vertraulich und d?rfen nur bestimmten Personen zug?nglich gemacht werden. Sofern Sie nicht zu den angegebenen Empf?ngern geh?ren, benachrichtigen Sie bitte unverz?glich den Absender. Der Inhalt darf weder an Dritte weitergegeben noch zu anderen Zwecken verwendet werden. Die Informationen d?rfen auch nicht auf einem Datentr?ger gespeichert oder auf einen Datentr?ger kopiert werden. -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.1.9-lib-storage-virtual_size_add_new-memory-leak.patch Type: text/x-patch Size: 873 bytes Desc: not available URL: From ben at indietorrent.org Thu Sep 6 20:56:58 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 13:56:58 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <50487684.8060404@Media-Brokers.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> Message-ID: <5048E3EA.80009@indietorrent.org> On 9/6/2012 6:10 AM, Charles Marcus wrote: > On 2012-09-05 6:20 PM, Ben Johnson wrote: > >> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu > 10.04. > > 1.2.9 is really old... you really need to upgrade to a recent/stable > version. Thanks, Charles. I do see your point. One of the challenges we face in this regard is that we're using a Long-Term-Support version of Ubuntu (10.04) and 1.2.9 is the latest package in the OS's repository. That said, we could upgrade manually, but this is a production server on which downtime must be minimized, and we all know how unexpected issues arise during installation (even when the procedure is tested in a closely equivalent development environment). >> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam > > That is for version 2.x (note the wiki2)... Noted; thank you. > I recommend going ahead and upgrading to the latest 2.1.9 and starting > over. > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle with this further. Thanks again, -Ben From lists at wildgooses.com Thu Sep 6 21:19:03 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 06 Sep 2012 19:19:03 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <5048E917.6030004@wildgooses.com> On 06/09/2012 18:56, Ben Johnson wrote: > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). I personally use (lightweight) virtualisation on any new machine, I really don't see any reason why NOT to. I would typically also setup my mounts such that the operating system is separate from "the data". This makes it easy to upgrade the OS/services, but without touching the data (test before/after on the same data for example) So in my situation I would boot a fairly small (gentoo in my case) virtual environment that runs only dovecot + postfix, it mounts the mail spools separately - I say "boot", but because I'm using linux-vservers, it's really a fancy chroot, and so the instance will start in 2-3 seconds (restarts are similarly near instant). I would upgrade by cloning this installation, upgrading it, testing it to bits, and then to make it live basically you swap this "machine" for the live machine. There are various ways it could be made near seamless, but in my situation I can bear a couple of seconds whilst I literally restart the "machine" Similarly I segregate all my services into a dozen or so "virtual machines", so DNS has it's own "machine" and so does logging, databases, almost every webservice gets its own virtual environment, etc. You could use a full blown vmware/kvm/etc if that floats your boat better, but the point remains it's so trivial to install, makes upgrades to trivial and massively decreases your downtime risk that it's very hard to find a reason NOT to do it... I haven't tried too hard to keep my instances tiny, so each is probably around 400-600MB in my case. However, if it were important this could easily be reduced to 10-100s MB each using various hardlink features. As you can see it's easy to snapshot a whole machine to manage upgrades/backups, etc This is more about infrastructure, but I honestly can't get over how many people are sitting on their hands shackled by "I'm on Debian xxx and I can't install any software newer than 5 years old"... It's so easy to escape from that trap...!! Good luck Ed W From ben at indietorrent.org Thu Sep 6 22:38:07 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 06 Sep 2012 15:38:07 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E917.6030004@wildgooses.com> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <5048E917.6030004@wildgooses.com> Message-ID: <5048FB9F.7030104@indietorrent.org> On 9/6/2012 2:19 PM, Ed W wrote: > On 06/09/2012 18:56, Ben Johnson wrote: >> >> On 9/6/2012 6:10 AM, Charles Marcus wrote: >>> On 2012-09-05 6:20 PM, Ben Johnson wrote: >>> >>>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >>> 10.04. >>> >>> 1.2.9 is really old... you really need to upgrade to a recent/stable >>> version. >> Thanks, Charles. I do see your point. One of the challenges we face in >> this regard is that we're using a Long-Term-Support version of Ubuntu >> (10.04) and 1.2.9 is the latest package in the OS's repository. >> >> That said, we could upgrade manually, but this is a production server on >> which downtime must be minimized, and we all know how unexpected issues >> arise during installation (even when the procedure is tested in a >> closely equivalent development environment). > > I personally use (lightweight) virtualisation on any new machine, I > really don't see any reason why NOT to. I would typically also setup my > mounts such that the operating system is separate from "the data". This > makes it easy to upgrade the OS/services, but without touching the data > (test before/after on the same data for example) Thanks for your valuable insights, Ed. That seems like a worthwhile approach. > So in my situation I would boot a fairly small (gentoo in my case) > virtual environment that runs only dovecot + postfix, it mounts the mail > spools separately - I say "boot", but because I'm using linux-vservers, > it's really a fancy chroot, and so the instance will start in 2-3 > seconds (restarts are similarly near instant). I would upgrade by > cloning this installation, upgrading it, testing it to bits, and then to > make it live basically you swap this "machine" for the live machine. > There are various ways it could be made near seamless, but in my > situation I can bear a couple of seconds whilst I literally restart the > "machine" > > Similarly I segregate all my services into a dozen or so "virtual > machines", so DNS has it's own "machine" and so does logging, databases, > almost every webservice gets its own virtual environment, etc. You could > use a full blown vmware/kvm/etc if that floats your boat better, but the > point remains it's so trivial to install, makes upgrades to trivial and > massively decreases your downtime risk that it's very hard to find a > reason NOT to do it... While I'm with you here, and I understand the theory (and practice, to some extent), doesn't all of this require a true, physical machine? We can't justify the expense associated with a physical machine in a hosted environment, so we're left with so-called VPSs. My understanding is that OpenVZ cannot be installed on a VPS (for seemingly obvious reasons -- namely, that the VPS is itself an OpenVZ container). > I haven't tried too hard to keep my instances tiny, so each is probably > around 400-600MB in my case. However, if it were important this could > easily be reduced to 10-100s MB each using various hardlink features. > As you can see it's easy to snapshot a whole machine to manage > upgrades/backups, etc > > > This is more about infrastructure, but I honestly can't get over how > many people are sitting on their hands shackled by "I'm on Debian xxx > and I can't install any software newer than 5 years old"... It's so easy > to escape from that trap...!! Perhaps easy, but not necessarily inexpensive. ;-) Thanks again for sharing the details of your strategy; I'll bear all of this in mind moving forward. > Good luck > > Ed W > > -Ben From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 01:12:58 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 00:12:58 +0200 Subject: [Dovecot] dovecotadm error In-Reply-To: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> References: <63ca9330f102f985843845d89e4cccda@mail.nettrust.net.nz> Message-ID: <20120906221258.GA9219@daniel.localdomain> Hi Michael, Michael wrote: > doveadm expunge -A mailbox Junk savedbefore 28d > doveadm(root): Error: User listing returned failure > > The mail log file says: > dovecot: auth-worker(18549): Error: sql: Iterate query failed: Table > 'system.users' doesn't exist (using built-in default iterate_query: > SELECT username, domain FROM users) > > dovecot-sql.conf: > iterate_query = SELECT username AS user FROM accounts Some examples in the wiki refer to "username", you could try this: iterate_query = SELECT username FROM accounts > though I have had to comment the following from dovecot.conf: > #iteratedb { > # args = /usr/local/etc/dovecot/dovecot-sql.conf > # driver = sql > #} > because it errors on dovecot start up. Your primary userdb refers to dovecot-sql.lda.conf, not dovecot-sql.conf. What does /usr/local/etc/dovecot/dovecot-sql.lda.conf look like and is there an iterate_query inside this file? Regards Daniel -- https://plus.google.com/103021802792276734820 From Bill at knoxvillechristian.org Fri Sep 7 01:50:19 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 06 Sep 2012 18:50:19 -0400 Subject: [Dovecot] thunderbird not connecting In-Reply-To: <1346732514245-37389.post@n4.nabble.com> References: <1346732514245-37389.post@n4.nabble.com> Message-ID: <504928AB.7040702@knoxvillechristian.org> Turn on more debugging with these in your configuration: auth_verbose = yes auth_debug = yes auth_debug_passwords = yes mail_debug = yes verbose_ssl = yes and then check the server logs after trying to login. Also, I'm curious why you masked your PRIVATE ip address and not your public one. Bill On 9/4/2012 12:21 AM, cc young wrote: > cannot get TB to recognize either pop3/s or imap/s server > > can connect just fine with: > > openssl s_client -connect ms1.myserver.net:993 > . login ... > > but trying with TB /var/log/mail.log gets: > > dovecot: pop3-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > dovecot: imap-login: Aborted login (no auth attempts): rip=223.205.150.234, > lip=xxx.xx.xx.xx > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/thunderbird-not-connecting-tp37389.html > Sent from the Dovecot mailing list archive at Nabble.com. From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 02:56:37 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 01:56:37 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 Message-ID: <20120906235637.GA11324@daniel.localdomain> Hi, I'm getting segfaults and unexpected disconnects from managesieve server, when the Thunderbird SIEVE extension tries to validate SIEVE scripts agains Pidgeonhole in Dovecot 2.1.8. The extension says: "Server terminated unexpectedly the connection, click on reconnect to try again." It can be reproduced by simply calling CHECKSCRIPT followed by a string: mail01:~# telnet 127.0.0.1 19200 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. "IMPLEMENTATION" "Sieve" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave" "NOTIFY" "mailto" "SASL" "PLAIN" "VERSION" "1.0" OK "Mailbox" AUTHENTICATE "PLAIN" "AGRwYXJ0aGV5QGV4YW1wbGUub3JnAGRwYXJ0aGV5" OK "Logged in." CHECKSCRIPT {6+} abcdef Connection closed by foreign host. The server interrupts the connection and the logs show the following: # dovecot.log Sep 7 01:40:46 dovecot: mailbox: mail: managesieve(dparthey at example.org): Fatal: master: service(managesieve): child 31356 killed with signal 11 (core dumped) # kern.log Sep 7 01:40:46 kernel: [1417105.954609] managesieve[31356]: segfault at 0 ip 00007f1c415c4876 sp 00007fffb3731f88 error 4 in libc-2.11.1.so[7f1c41543000+17a000] Here is the backtrace: mail01:~# gdb /usr/lib/dovecot/managesieve /var/tmp/core.managesieve.31356 Core was generated by `dovecot-mailbox/managesieve'. Program terminated with signal 11, Segmentation fault. #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 (gdb) bt full #0 0x00007f1c415c4876 in ?? () from /lib/libc.so.6 No symbol table info available. #1 0x000000000040b1c3 in sieve_storage_save_will_activate (ctx=0x257f1a0) at sieve-storage-save.c:328 _data_stack_cur_id = 4 scriptname = 0x25400ae "default.sieve" ret = #2 0x00000000004065b0 in cmd_putscript_finish_parsing (cmd=0x2575cb8) at cmd-putscript.c:206 ehandler = cpflags = sbin = errors = _data_stack_cur_id = 0 script = client = 0x2575c30 ctx = 0x2578180 args = 0x254bc68 ret = #3 0x0000000000406838 in cmd_putscript_continue_script (cmd=0x2575cb8) at cmd-putscript.c:423 all_written = client = 0x2575c30 ctx = 0x2578180 size = 39279968 #4 0x0000000000406caf in client_input_putscript (context=0x2575c30) at cmd-putscript.c:84 cmd = 0x2575cb8 __FUNCTION__ = "client_input_putscript" #5 0x00007f1c41d1d3e6 in io_loop_call_io (io=0x2575fd0) at ioloop.c:379 ioloop = 0x2548680 t_id = 2 #6 0x00007f1c41d1e46f in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 ctx = 0x25489f0 event = 0x2548a60 list = 0x2576020 io = 0x0 tv = {tv_sec = 18, tv_usec = 988469} msecs = ret = i = 0 call = false #7 0x00007f1c41d1d388 in io_loop_run (ioloop=0x2548680) at ioloop.c:398 No locals. #8 0x00007f1c41d09653 in master_service_run (service=0x2548530, callback=0x25400ae) at master-service.c:543 No locals. #9 0x00000000004096ce in main (argc=1, argv=0x2548370) at main.c:308 set_roots = {0x610d60, 0x0} login_set = {auth_socket_path = 0x2540088 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x409810 , failure_callback = 0x409290 } service_flags = storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-42-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = xxx imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } From stephan at rename-it.nl Fri Sep 7 03:14:50 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 07 Sep 2012 02:14:50 +0200 Subject: [Dovecot] Managesieve segfault with dovecot 2.1.8 In-Reply-To: <20120906235637.GA11324@daniel.localdomain> References: <20120906235637.GA11324@daniel.localdomain> Message-ID: <50493C7A.1070602@rename-it.nl> On 9/7/2012 1:56 AM, Daniel Parthey wrote: > Hi, > > I'm getting segfaults and unexpected disconnects from managesieve server, > when the Thunderbird SIEVE extension tries to validate SIEVE scripts > agains Pidgeonhole in Dovecot 2.1.8. > > The extension says: > "Server terminated unexpectedly the connection, click on reconnect to try again." > > It can be reproduced by simply calling CHECKSCRIPT followed by a string: This is a known and fixed problem: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/6ceeb6421231 I should make a release soon. Regards, Stephan. From Nico.Weinreich at drefa-msg.de Fri Sep 7 10:30:46 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Fri, 7 Sep 2012 09:30:46 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars Message-ID: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Hi! I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht" => Roundcube shows that messages folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht123" => Roundcube shows error that no messages where found folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung" => Roundcube shows that messages folder contains message /w subject "Aufl?sung der Frage" => search for "L?sung123" => Roundcube shows error saying: "Connection to storage server failed" Looking into /var/log/mail.log shows: Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4478a) [0 x7f065ba8e78a] -> /usr/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f065ba8e7d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f065ba655ef ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a3cb) [0x7f065bd6f3cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a8aa) [0x7f065bd6f8 aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x78641) [0x7f065bd4d641] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_fore ach+0x3c) [0x7f065bd4d7bc] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header+0x29) [0x7f065ba78eb9] -> /usr/lib/dovecot/libdovecot-st orage.so.0(+0x9a0cf) [0x7f065bd6f0cf] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9b160) [0x7f065bd70160] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_storage_search_next_nonblock+0x48) [0x7f065bd706e8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonbl ock+0x27) [0x7f065bd51a07] -> dovecot/imap() [0x41668f] -> dovecot/imap(imap_search_start+0xd8) [0x416ad8] -> dovecot/imap(cmd_sort+0x1e6) [ 0x40e636] -> dovecot/imap(command_exec+0x3d) [0x41157d] -> dovecot/imap() [0x4104fe] -> dovecot/imap(client_handle_input+0x135) [0x410805] - > dovecot/imap(client_input+0x5f) [0x41112f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f065ba9a926] -> /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f065ba9b95f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f065ba9a8c8] -> /usr/li b/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f065ba87053] -> dovecot/imap(main+0x2a4) [0x419354] -> /lib/libc.so.6(__libc_start_ma in+0xfd) [0x7f065b706c8d] -> dovecot/imap() [0x408e79] Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? Regards From ml at cibest.com Fri Sep 7 11:46:42 2012 From: ml at cibest.com (Cibest ML) Date: Fri, 07 Sep 2012 10:46:42 +0200 Subject: [Dovecot] Wrong quota calculation Message-ID: <5049B472.8060407@cibest.com> Hi, We've a strange quota behavior on our dovecot servers. It seems that it's somehow doubling the real used storage (or could be something like twice the inbox size + size of other folders, hard to know for sure). It was initial setup with the maildir++ system. Now it's been configured with dict and mysql table but the result is the same. Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used As example, let's use mailbox at domain.com. If I check in the quota table, I'll find : - size : 697151354 - number of files : 1510 Now if I check the Maildir folder on the filesystem, I'll find : - size : 347960000 - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, not 100% accurate) Another mail box Reported by dovecot : 503446986 1081 Filesystem : 248592000 598 Does anyone has an idea of what could be possibly wrong. We've checked & double-checked everything and nothing seems to be wrong (no dsync used). -- Regards S?bastien From amateo at um.es Fri Sep 7 13:11:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:11:45 +0200 Subject: [Dovecot] best way to activate quota Message-ID: <5049C861.2030904@um.es> Hello, I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). What do you think is the best to activate it? I have think that I could use doveadm commands to calculate an initial quota (on low load periods) and then activate quota, hoping that although quota is not correct, it would be used. Should this work? From robert at schetterer.org Fri Sep 7 13:28:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 07 Sep 2012 12:28:45 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: <5049CC5D.7070005@schetterer.org> Am 07.09.2012 12:11, schrieb Angel L. Mateo: > Hello, > > I'm planning to activate quota control in dovecot, with maildir > quota backend. I have about 70k users in my system directed to 4 backend > servers (with a director to ensure that a user is always directed to the > same server). sounds good, first look > > I have tried to activate it in one of my nodes. The problem is that > load of it has increase a lot, as much as the system was unusable > (maildir is in nfs storage, with indexes in local disks). that might not be the optimal storage solution > > What do you think is the best to activate it? yes ,until its better investigated, cause your systems shouldnt get unusable > > I have think that I could use doveadm commands to calculate an > initial quota (on low load periods) and then activate quota, hoping that > although quota is not correct, it would be used. Should this work? > > with 70 k users , you should be able to hire professional help from timo and/or other dovecot specialists however you ever will have to post your config/logs/general-setup-big-picture if asking for help -- Best Regards MfG Robert Schetterer From amateo at um.es Fri Sep 7 13:41:40 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 07 Sep 2012 12:41:40 +0200 Subject: [Dovecot] Removing specific entry in user/auth cache In-Reply-To: <1341388872.2689.3.camel@innu> References: <1340865829.25551.64.camel@innu> <42916718-6B7E-4632-8C61-AA8FE64D850E@iki.fi> <1341388872.2689.3.camel@innu> Message-ID: <5049CF64.8090908@um.es> El 04/07/12 10:01, Timo Sirainen escribi?: > On Fri, 2012-06-29 at 05:01 +0300, Timo Sirainen wrote: >> and for v2.1 a bit kludgy way: >> >> doveadm auth [] >> doveadm auth cache flush [] > > Done: http://hg.dovecot.org/dovecot-2.1/rev/007bf0047ab0 > http://hg.dovecot.org/dovecot-2.1/rev/1093c74f54af > Hello, After some time I have updated my system to 2.1.9 which includes this patch but I have doubts it is working. I have changed an attribute for one of my users (his home directory) so I run: root at myotis33:~# doveadm auth cache flush 2 cache entries flushed but, then, when I run "doveadm user " I've got the old information, not the updated one. I had to reload dovecot to get the information correctly reloaded. From hmoreno at gmv.com Fri Sep 7 13:43:19 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Fri, 7 Sep 2012 10:43:19 +0000 Subject: [Dovecot] Quota less mailbox with mdbox Message-ID: Good morning everyone, I have a little doubt about setting a quotaless mailbox with mdbox. Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. The migration is simple and easy, but there are some configuration settings we cannot set to work properly. In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. This is working like a charm. Dovecot.conf: ... namespace { separator = . prefix = inbox = yes type = private } namespace { inbox = no location = prefix = INBOX. separator = . hidden = yes type = private list = no } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } ... plugin/quota = maildir:_backup:ns= ... Dovecot-ldap.conf: ... pass_filter = (&(objectClass=qmailUser)(uid=%u)(accountActive=TRUE)) pass_attrs = uid=user,userPassword=password, mailQuotaSize=quota_rule=*:bytes=%$ user_attrs = mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ user_filter = (&(objectClass=qmailUser)(|(mailAlternateAddress=%u)(mail=%u)(uid=%u))(accountActive=TRUE)) ... When migrated to mdbox and changing the configuration, we don't know how to get it work. We have also tried to follow the instructions from the wiki of dovecot2 without success. Could anyone give us a clue please? If additional information is needed, please let me know. Thank you very much in advanced. Kind regards. ________________________________ [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\logo_gmv.gif] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_blog.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_facebook.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_twitter.gif] [Descripci?n: C:\Documents and Settings\hhmb\Datos de programa\Microsoft\Firmas\icon_youtube.gif] P Please consider the environment before printing this e-mail. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From kiwi at oav.net Fri Sep 7 14:30:42 2012 From: kiwi at oav.net (Xavier Beaudouin) Date: Fri, 7 Sep 2012 13:30:42 +0200 Subject: [Dovecot] dsync mirror and sieve ? Message-ID: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Hi there, I am currently doing some extensive test with dsync, but it appear that : dsync -u maildir:/path/to/new/Maildir doesn't seems to sync sieves rules. Is there a good way to sync sieves as well ? :p Regards, Xavier From mhinch at hinchfamily.com Fri Sep 7 14:39:24 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Fri, 07 Sep 2012 06:39:24 -0500 Subject: [Dovecot] dovecot/log not running Message-ID: <5049DCEC.5050008@hinchfamily.com> I'm running 2.1.7 as downloaded/installed by Debian (and then appropriately setup for my config, ...). It almost works. I can send/receive email from thunderbird, squirrelmail, ... However there are two anomalies: 1. dovecot/log is not running. After restarting dovecot (and waiting 30 seconds), I have this in the log files: Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 After 30 seconds, a "ps -ef | grep dovecot" yields: dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil root 25148 20223 0 18:55 ? 00:00:00 dovecot/config dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). A "doveadm log find" dumps this in the log: Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) However, further debugging that I try to do (see next problem) does not result in ANY further input to the log files, even though I have all known logging turned on. So I *think* that logging is not actually running. If I can get that started, then I'll take a crack at the next problem myself ... 2. However, as long as I have your attention, my next problem is that it seems like dovecot/imap-login hangs(?). After a few hours, I have something like: # ps -ef | grep dovecot dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth root 13277 31186 0 17:47 ? 00:00:00 dovecot/config dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil with exactly 100 imap-login processes. I'm the only meaningful user on the machine (unless somebody is imap DoS-ing me), so this does not make much sense. Obviously thunderbird is polling the machine and each time is leaving around a imap-log process. As far as I can tell, thunderbird successfully logs in (i.e., I get mail) though I can't be sure whether these are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? After a while of course, dovecot will not start any new imap-login processes so Thunderbird cannot get any more mail until I restart dovecot. Then everything is OK for another 12 hours or so. Ideas on either problem? -Mark From CMarcus at Media-Brokers.com Fri Sep 7 14:59:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 07 Sep 2012 07:59:44 -0400 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <5049E1B0.2030602@Media-Brokers.com> On 2012-09-07 6:43 AM, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. If you're going to migrate to mdbox, you'd be much better off if you upgraded to the latest 2.1.x... Mdbox is much better supported in the current stable builds. -- Best regards, Charles From trashcan at odo.in-berlin.de Fri Sep 7 15:16:14 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 07 Sep 2012 14:16:14 +0200 Subject: [Dovecot] =?utf-8?q?dsync_mirror_and_sieve_=3F?= In-Reply-To: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> References: <4E61E4C5-BB02-4D93-88DE-7E39D912BBDC@oav.net> Message-ID: <0ae24afcca736872e3795e930612cb67@mx1.enfer-du-nord.net> Hi -- Xavier Beaudouin wrote: > I am currently doing some extensive test with dsync, but it appear > that : > > dsync -u maildir:/path/to/new/Maildir > > doesn't seems to sync sieves rules. > > Is there a good way to sync sieves as well ? :p I'm using Unison to sync my user's dedicated sievedirs. Regards, Michael From tth at rfa.cz Fri Sep 7 18:34:34 2012 From: tth at rfa.cz (TeTeHacko) Date: Fri, 07 Sep 2012 17:34:34 +0200 Subject: [Dovecot] pop3 migration In-Reply-To: <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> References: <503632C7.2070008@rfa.cz> <984B7F63-322D-430E-B495-88ED2274728A@iki.fi> Message-ID: <504A140A.6070000@rfa.cz> Probably on server side ... the problem was only on big mailboxes which is used by IMAP, so there is no problem and we have already migrated all to dovecot. ;) Thanks for your reply. On 31.8.2012 15:46, Timo Sirainen wrote: > On 23.8.2012, at 16.40, TeTeHacko wrote: > >> I have problems with pop3 migration, the server disconect the dovesync when >> fetching mails after 5 minutes with this error in server log: >> >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6548 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 47522 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6549 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 355289 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6550 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 9638 octets >> xx.xx.xx.xx [28F4] 11:54:05 <<< RETR 6551 >> xx.xx.xx.xx [28F4] 11:54:05 >>> +OK 2188 octets >> xx.xx.xx.xx [28F4] 11:59:05 >>> -ERR Command unrecognized: "" >> xx.xx.xx.xx [28F4] 11:59:05 *** 6551 477693894 00:12:55 ERROR >> xx.xx.xx.xx [28F4] 11:59:05 Disconnected > > Looks like a bug. But in Dovecot POP3 client or the POP3 server? Apparently either Dovecot sends an empty line after RETR or the server thinks Dovecot sends it. Can you get the actual TCP traffic logs? and/or set pop3c_rawlog_dir and see if it's there. > From trashcan at odo.in-berlin.de Fri Sep 7 22:22:48 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 7 Sep 2012 21:22:48 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? Message-ID: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Hi -- Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I just want to be sure that I haven't overseen such a tool/functionality in the documentation. Did I or didn't I? Thanks and regards, Michael From Ralf.Hildebrandt at charite.de Fri Sep 7 22:30:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 7 Sep 2012 21:30:32 +0200 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907193032.GD11025@charite.de> * Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot (accepted by postfix), temporarily. Now, I would like to know if there is a way to make a "doveadm kick -A" a permanent disconnect until one would allow reconnects, again? This should be done *without* shutting down dovecot or *extensive* re-configuration. A "doveadm block -u -m 'sorry: running maintenance, please, come back later'" alike tool/funtionality would be ideal. > > Background: I would like to avoid any modification of my user's maildirs by a running dovecot server while modifying maildir formats, mdbox_rotate_size setting, and alike, where I would need doveadm/dsnyc access to userdb. (I know there a ways to achieve this without downtimes, and I have already done so.) I think this can be done with post-login scripting, where a script checks if the user is allowed to login. http://wiki.dovecot.org/PostLoginScripting -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From silent_wumpus at yahoo.com Fri Sep 7 22:32:38 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Fri, 7 Sep 2012 12:32:38 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method Message-ID: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From rostetter at mail.utexas.edu Fri Sep 7 23:10:30 2012 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Fri, 07 Sep 2012 15:10:30 -0500 Subject: [Dovecot] Disconnect users for a distinct period of time? In-Reply-To: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> References: <60295D41-33D4-4E67-892B-83FF3A4CBF45@odo.in-berlin.de> Message-ID: <20120907151030.44362ukqt9598wqo@mail.ph.utexas.edu> Quoting Michael Grimm : > Hi -- > > Today, I've learned who to disable incoming mail delivery to dovecot > (accepted by postfix), temporarily. Now, I would like to know if > there is a way to make a "doveadm kick -A" a permanent disconnect > until one would allow reconnects, again? This should be done > *without* shutting down dovecot or *extensive* re-configuration. A > "doveadm block -u -m 'sorry: running maintenance, please, come > back later'" alike tool/funtionality would be ideal. What I've been doing so far, is I configure dovecot 2.1 with a deny-hosts file (in auth-deny.conf.ext) and then I can append a name to that file to block them, and remove them to allow them back in. Since this file is a simple text file with one username/address per line, it is very easy to manage. No idea if that would work for you, since your usage is for a different reason than mine. -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From daniel.parthey at informatik.tu-chemnitz.de Fri Sep 7 23:46:50 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 22:46:50 +0200 Subject: [Dovecot] about postlogin in dovecot2 In-Reply-To: <1346732123552-37388.post@n4.nabble.com> References: <1344636852567-5993.post@n4.nabble.com> <1344924660889-37062.post@n4.nabble.com> <06712416-A68F-4153-9F07-4412B43DA3E8@iki.fi> <1344925564701-37064.post@n4.nabble.com> <20120814230403.GA11246@daniel.localdomain> <1346732123552-37388.post@n4.nabble.com> Message-ID: <20120907204650.GA9602@daniel.localdomain> bmalik wrote: > when i login to the server using pop3s, i see in the dovecot log; > > @4000000050457dd9179c653c pop3-login: Info: Login: user=, > method=DIGEST-MD5, rip=192.168.12.220, lip=192.168.12.115, mpid=87395, TLS, > session= > > if i add my mail address into the users-with-forced-encryption file, > i would not reach my account. > > i get an error; > @4000000050457dd9183f542c pop3(test at byserver.net): Info: Post-login script > denied access to user test at byserver.net > whereas i connect to the account via pop3s > if i remove my mail address from the users-with-forced-encryption > there is no problem. > > my postlogin_pop3.sh; > #!/usr/local/bin/bash > if [ "$SECURED" == "secure" ] > then > :# handle secure connections here > else > :# handle insecure connections here > if grep -q "$USER" in /usr/local/etc/dovecot/users-with-forced-encryption > then > printf "* NO [ALERT] You are not allowed to connect without > encryption\r\n" > exit 0 > fi > fi > exec "$@" > ########### > there is also no problem related to SSL certificate > what can be the issue ? Can you log the value of "$SECURED" to a logfile (using "logger" tool), and log a message in each of the conditional branches of your script, so we can see which branch gets called? Which error message do you get when you manually try to log into POP3S? openssl s_client -connect ip:port USER username PASS password QUIT Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 00:08:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 7 Sep 2012 23:08:33 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1346745602879-37394.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> Message-ID: <20120907210833.GA10535@daniel.localdomain> bmalik wrote: > I upgraded dovecot from 1.1 to 2.1.8 > > pop3/imap works but i have a problem. > > i use pop3 protocol on outlook 2010 for my account. and i set keep my mails > on the server for 5 days or until i delete them. > These settings were working old dovecot1.1. But when i upgraded it to 2.1.8. > if i even delete a mail from outlook the server doesn't delete or old mails > are not deleted. Dovecot does not remove expunged mails from mdbox automatically. You can run doveadm purge via cron to remove unreferenced messages: http://wiki2.dovecot.org/Tools/Doveadm/Purge Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Sep 8 02:38:44 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 8 Sep 2012 01:38:44 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5049B472.8060407@cibest.com> References: <5049B472.8060407@cibest.com> Message-ID: <20120907233844.GA12996@daniel.localdomain> Cibest ML wrote: > We've a strange quota behavior on our dovecot servers. > It seems that it's somehow doubling the real used storage (or could > be something like twice the inbox size + size of other folders, hard > to know for sure). > It was initial setup with the maildir++ system. Now it's been > configured with dict and mysql table but the result is the same. > > Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used > As example, let's use mailbox at domain.com. > If I check in the quota table, I'll find : > - size : 697151354 > - number of files : 1510 > > Now if I check the Maildir folder on the filesystem, I'll find : > - size : 347960000 > - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, > not 100% accurate) > > Another mail box > Reported by dovecot : 503446986 1081 > Filesystem : 248592000 598 > > Does anyone has an idea of what could be possibly wrong. We've > checked & double-checked everything and nothing seems to be wrong > (no dsync used). What happens after quota resync? Does the quota change? What happens to the "doveadm quota get" output after LMTP/LDA delivery of a mail with a fixed-size, let's say one mail with 1 MB? How does your setup look like? Mailbox+Director Proxy or only Mailboxes? What is your doveconf -n output of mailbox (and possibly director)? Regards Daniel -- https://plus.google.com/103021802792276734820 From dieterknopf at googlemail.com Sat Sep 8 08:11:34 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sat, 8 Sep 2012 07:11:34 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering Message-ID: Hello, is there any update about the sieve command line tool? I really need some way to refilter my mails without loosings flags and without moving all messages in a different folder. This is possible with procmail via direct access to the file system without the need of getmail. Thanks From robert at schetterer.org Sat Sep 8 09:14:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 08:14:27 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AE243.60207@schetterer.org> Am 08.09.2012 07:11, schrieb Dieter Knopf: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. > > This is possible with procmail via direct access to the file system > without the need > of getmail. > > Thanks > look http://wiki2.dovecot.org/Pigeonhole/Sieve there is some special sieve stuff to filter against external progs i.e. procmail -- Best Regards MfG Robert Schetterer From bmalik at ihlas.net.tr Sat Sep 8 10:25:58 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 00:25:58 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <20120907210833.GA10535@daniel.localdomain> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> Message-ID: <1347089158969-37479.post@n4.nabble.com> But i use Maildir ? is there a solution ? Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i got duplicate mails which i have kept for some days on the server. how can i prevent this problem? -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html Sent from the Dovecot mailing list archive at Nabble.com. From stephan at rename-it.nl Sat Sep 8 10:39:06 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 08 Sep 2012 09:39:06 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: Message-ID: <504AF61A.9090906@rename-it.nl> On 9/8/2012 7:11 AM, Dieter Knopf wrote: > Hello, > > is there any update about the sieve command line tool? > > I really need some way to refilter my mails without loosings flags and without > moving all messages in a different folder. For Pigeonhole v0.3 it is part of the normal release as the sieve-filter command. I still haven't received much feedback on it though, so make sure your backups are in order before you give it a try and please report back your findings. Regards, Stephan. From robert at schetterer.org Sat Sep 8 16:10:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:10:54 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347089158969-37479.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> Message-ID: <504B43DE.8050008@schetterer.org> Am 08.09.2012 09:25, schrieb bmalik: > But i use Maildir ? > > is there a solution ? > > Meanwhile, after dovecot to 2.1.8, when i login to the new pop3 server, i > got duplicate mails which i have kept for some days on the server. > how can i prevent this problem? > > > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37479.html > Sent from the Dovecot mailing list archive at Nabble.com. > pop3 feature leave mails on the servers works i.e with thunderbird with a local list in Tb what is allready download for compare so its not only a server feature, so you will allways download all mails new if using a new client without that list, or you changed something on the server ( i.e X-UIDL changed ) and existing list isnt longer build like before see i.e http://kb.mozillazine.org/Popstate.dat however i had some problems with pop3 leave mails on the server with virtual plugin, but that was fixed and works with setup right way -- Best Regards MfG Robert Schetterer From robert at schetterer.org Sat Sep 8 16:12:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 08 Sep 2012 15:12:53 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: <504B4455.8090600@schetterer.org> Am 08.09.2012 09:39, schrieb Stephan Bosch: > On 9/8/2012 7:11 AM, Dieter Knopf wrote: >> Hello, >> >> is there any update about the sieve command line tool? >> >> I really need some way to refilter my mails without loosings flags and >> without >> moving all messages in a different folder. > > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make > sure your backups are in order before you give it a try and please > report back your findings. > > Regards, > > Stephan. > only for info last time i tried external sieve stuff, it worked with procmail, but i stopped doing something with it -- Best Regards MfG Robert Schetterer From h.reindl at thelounge.net Sat Sep 8 16:16:06 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:16:06 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <504B4516.8040908@thelounge.net> Am 08.09.2012 15:10, schrieb Robert Schetterer: > pop3 feature leave mails on the servers works i.e with thunderbird > with a local list in Tb what is allready download for compare > so its not only a server feature, so you will allways download all mails > new if using a new client without that list, or you changed something > on the server ( i.e X-UIDL changed ) and existing list isnt longer build > like before > > see i.e > > http://kb.mozillazine.org/Popstate.dat > > however i had some problems with pop3 leave mails on the server > with virtual plugin, but that was fixed and works with setup right way and that is why every mail-client has a option "delete messages after x days from server" but users are refusing to spent some minutes of their time looking what options software offers for wondering years later -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From bmalik at ihlas.net.tr Sat Sep 8 16:43:18 2012 From: bmalik at ihlas.net.tr (bmalik) Date: Sat, 8 Sep 2012 06:43:18 -0700 (PDT) Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <504B43DE.8050008@schetterer.org> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> Message-ID: <1347111798522-37484.post@n4.nabble.com> i only upgraded dovecot from 1.x to 2.1.x. i didn't change any settings over server. after dovecot service restart, my keep mails on the server, i got again to the outlook. that's my test server but i am going to migrate soon. but i don't want to get duplicate mails. How can i prevent ? Thanks -- View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html Sent from the Dovecot mailing list archive at Nabble.com. From h.reindl at thelounge.net Sat Sep 8 16:47:41 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 08 Sep 2012 15:47:41 +0200 Subject: [Dovecot] problem about pop3 using dovecot In-Reply-To: <1347111798522-37484.post@n4.nabble.com> References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B4C7D.108@thelounge.net> Am 08.09.2012 15:43, schrieb bmalik: > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? again: you can NOT POP3 and leave on server depends on the client having a list of message-UUIDs which are already received and it is not uncommon that they are changing due major upgrades on the server DO NOT LEAVE MAILS FOREVER ON THE SERVER this is the only solution and will ever be that is why the client offers "delete after X days from server" smarter clients like thunderbird are defaulting to 14 days not so smart client sleaves their messages forever on the server this is wasting ressources, leads to your troubles and that is why users should learn to look at the options of their sofwtare and try to understand basics who things are working or live with the resulsts if not doing so -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ml at cibest.com Sat Sep 8 18:07:26 2012 From: ml at cibest.com (Cibest ML) Date: Sat, 08 Sep 2012 17:07:26 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120907233844.GA12996@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> Message-ID: <504B5F2E.7070202@cibest.com> On 08/09/2012 01:38, Daniel Parthey wrote: > Cibest ML wrote: >> We've a strange quota behavior on our dovecot servers. >> It seems that it's somehow doubling the real used storage (or could >> be something like twice the inbox size + size of other folders, hard >> to know for sure). >> It was initial setup with the maildir++ system. Now it's been >> configured with dict and mysql table but the result is the same. >> >> Dovecot version : 2.1.7 (from squeeze-backports). Maildir format is used >> As example, let's use mailbox at domain.com. >> If I check in the quota table, I'll find : >> - size : 697151354 >> - number of files : 1510 >> >> Now if I check the Maildir folder on the filesystem, I'll find : >> - size : 347960000 >> - number of files : 824 (using find . -type f|grep -v dovecot|wc -l, >> not 100% accurate) >> >> Another mail box >> Reported by dovecot : 503446986 1081 >> Filesystem : 248592000 598 >> >> Does anyone has an idea of what could be possibly wrong. We've >> checked & double-checked everything and nothing seems to be wrong >> (no dsync used). > What happens after quota resync? Does the quota change? > > What happens to the "doveadm quota get" output after LMTP/LDA delivery > of a mail with a fixed-size, let's say one mail with 1 MB? > > How does your setup look like? Mailbox+Director Proxy or only Mailboxes? > > What is your doveconf -n output of mailbox (and possibly director)? > > Regards > Daniel Hi Daniel, Thanks for your answer Here are some tests to show the problem. The test mailbox is initialy empty. All described actions are done in a sequencial way (test mail with an attachement of ~2Mo) - Message delivered by LMTP Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - quota recalc Quota name Type Value Limit User quota STORAGE 5307 524288 User quota MESSAGE 2 - - move to draft + compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move to subfolder "test" in inbox User quota STORAGE 2653 524288 User quota MESSAGE 1 - - move message back to inbox Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - - compact Quota name Type Value Limit User quota STORAGE 2653 524288 User quota MESSAGE 1 - - recalc quota Quota name Type Value Limit User quota STORAGE 5306 524288 User quota MESSAGE 2 - No dovecot director is used at the moment (it's planned). There's one server handling incoming mails (postfix + LMTP), one for IMAP/POP (dovecot) & one for email sending (postfix). dovecot -n output of incoming mails server # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.4.1 Debian 6.0.5 ext4 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_gid = 1500 mail_home = /data/hosting/mail%h/%d/%n mail_location = maildir:/data/hosting/mail%h/%d/%n/Maildir mail_nfs_storage = yes mail_plugins = " quota autocreate" mail_uid = 1500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Inbox autocreate2 = Trash autocreate3 = Sent autocreate4 = Spam autocreate5 = Drafts autosubscribe = Inbox autosubscribe2 = Trash autosubscribe3 = Sent autosubscribe4 = Spam autosubscribe5 = Drafts quota = dict:User quota::noenforcing:proxy::quota quota_rule = *:storage=512M quota_rule2 = Trash:storage=+100M quota_warning = storage=90%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /data/hosting/mail%h/%d/%n/sieve/.dovecot.sieve sieve_before = /etc/dovecot/globalsieve sieve_dir = /data/hosting/mail%h/%d/%n/sieve/ } protocols = " lmtp sieve" quota_full_tempfail = yes service auth { unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 10.3.65.71 127.0.0.1 ::1 port = 2055 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } ssl_cert = References: <1346745602879-37394.post@n4.nabble.com> <20120907210833.GA10535@daniel.localdomain> <1347089158969-37479.post@n4.nabble.com> <504B43DE.8050008@schetterer.org> <1347111798522-37484.post@n4.nabble.com> Message-ID: <504B80A0.9010105@schetterer.org> Am 08.09.2012 15:43, schrieb bmalik: > > > i only upgraded dovecot from 1.x to 2.1.x. > i didn't change any settings over server. after dovecot service restart, my > keep mails on the server, i got again to the outlook. that's my test server > but i am going to migrate soon. but i don't want to get duplicate mails. > How can i prevent ? > Thanks > dont use pop3, use imap > > > -- > View this message in context: http://dovecot.2317879.n4.nabble.com/problem-about-pop3-using-dovecot-tp37394p37484.html > Sent from the Dovecot mailing list archive at Nabble.com. > -- Best Regards MfG Robert Schetterer From haldir.junk at gmail.com Sat Sep 8 23:55:20 2012 From: haldir.junk at gmail.com (Eric Sisolak) Date: Sat, 8 Sep 2012 16:55:20 -0400 Subject: [Dovecot] Vpopmail Dynamic Authentication Module In-Reply-To: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> References: <8992E740-591B-4572-AE89-5260D40CBB8F@iki.fi> Message-ID: On Fri, Aug 31, 2012 at 9:49 AM, Timo Sirainen wrote: > On 24.8.2012, at 16.42, Eric Sisolak wrote: > > > I had to modify the command to build the module, but was able to > > successfully build it: > > gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL > > -I../.. -I../lib -I/home/vpopmail/include -I/home/vpopmail/lib > > userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so > > -L/home/vpopmail/lib/libvpopmail.a > > -L means a path, pointing it to .a file doesn't do anything useful. By > giving the .a file directly probably also fails if libvpopmail.a hasn't > been built with -fPIC. > > > /usr/lib64/dovecot/auth/vpopmail.so: undefined symbol: vclose > > And that's the reason this fails, because libvpopmail isn't included in > vpopmail.so. > > Ah that was it, I was misunderstanding -l and -L. I got it to work with: gcc -shared -fPIC -DHAVE_CONFIG_H -DUSERDB_VPOPMAIL -DPASSDB_VPOPMAIL -I../.. -I../lib -I/home/vpopmail/include -L/home/vpopmail/lib userdb-vpopmail.c passdb-vpopmail.c -o vpopmail.so -lvpopmail Thanks! From dieterknopf at googlemail.com Sun Sep 9 06:42:15 2012 From: dieterknopf at googlemail.com (Dieter Knopf) Date: Sun, 9 Sep 2012 05:42:15 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: <504AF61A.9090906@rename-it.nl> References: <504AF61A.9090906@rename-it.nl> Message-ID: 2012/9/8 Stephan Bosch : > For Pigeonhole v0.3 it is part of the normal release as the sieve-filter > command. I still haven't received much feedback on it though, so make sure > your backups are in order before you give it a try and please report back > your findings. Thanks for the info. I'm not really understanding that tool :-( I tried this: sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep But i only get an error: sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' What's the problem? ._REFILTER_ is a standard IMAP subfolder. Thanks From manu at netbsd.org Sun Sep 9 10:17:46 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Sun, 9 Sep 2012 09:17:46 +0200 Subject: [Dovecot] Auth worker max line size In-Reply-To: <19A4E62F-42A0-4FDD-AD68-31C1B26B9E65@iki.fi> Message-ID: <1kq5dlx.15gv4r5j58ya7M%manu@netbsd.org> Timo Sirainen wrote: > Couldn't you change the client to use AUTHENTICATE PLAIN command instead? > The buffer wouldn't be a problem then.. Sorry for the delay, I missed the reply. That is not an option, as the client is not SASL capable. --- src/lib-master/master-auth.h.orig +++ src/lib-master/master-auth.h @@ -13,9 +13,9 @@ /* Authentication client process's cookie size */ #define MASTER_AUTH_COOKIE_SIZE (128/8) /* LOGIN_MAX_INBUF_SIZE should be based on this.*/ -#define MASTER_AUTH_MAX_DATA_SIZE 1024 +#define MASTER_AUTH_MAX_DATA_SIZE 4096 #define MASTER_AUTH_ERRMSG_INTERNAL_FAILURE \ "Internal error occurred. Refer to server log for more information." -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu at netbsd.org From stephan at rename-it.nl Sun Sep 9 11:48:47 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 09 Sep 2012 10:48:47 +0200 Subject: [Dovecot] Sieve command line tool status for Refiltering In-Reply-To: References: <504AF61A.9090906@rename-it.nl> Message-ID: <504C57EF.4090206@rename-it.nl> On 9/9/2012 5:42 AM, Dieter Knopf wrote: > 2012/9/8 Stephan Bosch : > >> For Pigeonhole v0.3 it is part of the normal release as the sieve-filter >> command. I still haven't received much feedback on it though, so make sure >> your backups are in order before you give it a try and please report back >> your findings. > Thanks for the info. > > I'm not really understanding that tool :-( > > I tried this: > sieve-filter -u foo at foo.tld main.sieve /home/vmail/foo/foo/._REFILTER_/ keep > > But i only get an error: > sieve-filter(root): Fatal: Unknown namespace for source mailbox '...' > > What's the problem? ._REFILTER_ is a standard IMAP subfolder. Mailboxes are not specified in terms of filesystem paths; these are specified in much the same way as for IMAP (albeit in UTF8 where relevant) and doveadm mailbox. So you should try: sieve-filter -u foo at foo.tld main.sieve _REFILTER_ keep The documentation is part of the problem I guess. I've adjusted the man page to specify the mailbox naming explicitly: Regards, Stephan. From roundcube222 at alaadin.org Sun Sep 9 20:51:30 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Sun, 09 Sep 2012 20:51:30 +0300 Subject: [Dovecot] ChgrpNoPerm issue Message-ID: <1b9365b674565d8a794678f3a2858712@Coptics.org> I have dovecot version 2.0.9 I am confused about this article http://wiki2.dovecot.org/Errors/ChgrpNoPerm my mail_location = mbox:~/mymail:INBOX=/var/mail/%u So my inbox is at var/mail/ and my imap folder is /home/user/mymail The problem is /var/mail (Owner is useraccount and the group us mail) , and here comes the problem Dovecot keeps trying to chown the imap folder inside the homedirectory with user:mail account and since And this is the reason of error that appears in my log file. The option mail_access_groups=mail solve the problem.. but I read it is not secure.. With my current configutation, users login with imap to read mail , can they use any the mail_access_groups=mail and read other poeple mail ? does my configutation is a shard mailbox and could be unsecure.. my main quesion , for my current configuration , can i use mail_access_groups=mail safely or can some user use this to read other people mail ? attached my configuration. why didnot have this issue with uw-imapd? and why dovecot try to chown the .imap folder with the mail group ? dovecot -n output is # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.2.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <20120909213052.GA11444@daniel.localdomain> Cibest ML wrote: > On 08/09/2012 01:38, Daniel Parthey wrote: > >Cibest ML wrote: > >>We've a strange quota behavior on our dovecot servers. > >>It seems that it's somehow doubling the real used storage (or could > >>be something like twice the inbox size + size of other folders, hard > >>to know for sure). > >>It was initial setup with the maildir++ system. Now it's been > >>configured with dict and mysql table but the result is the same. > Here are some tests to show the problem. The test mailbox is > initialy empty. All described actions are done in a sequencial way > (test mail with an attachement of ~2MB) > > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems ok. > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Strange. There must be some issue with quota recalculation. Maybe there are two instances of the message in the maildir? How large is the actual maildir measured by diskusage (du) command and where are the messages stored in the filesystem? > - move to draft + compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems absolutely correct. Message is copied, deleted and removed from the old directory. > - recalc > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - Seems after compacting the folders there is only one instance of the message. > - move to subfolder "test" in inbox > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > - move message back to inbox > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - The message gets copied and deleted, so there are 2 messages. > - compact > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - After compaction the "copied+deleted" message gets finally removed. > - recalc quota > Quota name Type Value Limit > User quota STORAGE 5306 524288 > User quota MESSAGE 2 - Very strange. The removed message "reappeared"? Are you using some network filesystem and possibly experience filesystem caching effects? In this case a dovecot director would help. Regards Daniel -- https://plus.google.com/103021802792276734820 From lgb at lgb.hu Mon Sep 10 14:33:08 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Mon, 10 Sep 2012 13:33:08 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" Message-ID: <20120910113307.GG14437@vega.lgb.hu> Hi, I'm trying to use dsync (dovecot version 2.1.9) for IMAP only per-user migration from other IMAP servers to my shiny new one. One worked quite well, but there is a problem with an old IMAP server: dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Looking up IP address dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Connecting to 192.168.10.100:143 dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' dsync(migtest at office): Debug: imapc(imap-sunw-old.servers.intra:143): Disconnected dsync(migtest at office): Error: imapc: Command failed: Disconnected from server dsync(migtest at office): Error: user migtest at office: Initialization failed: Initializing mail storage from mail_location setting failed: imapc: LIST failed: Internal error occurred. Refer to server log for more information. [2012-09-10 11:49:02] dsync(migtest at office): Fatal: User init failed What can cause this? "By hand" test with telnet'ing says: dovecot-test:~$ telnet imap-sunw-old.servers.intra 143 Trying 192.168.10.100... Connected to imap-sunw-old.servers.intra. Escape character is '^]'. * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging Server 6.3-5.02 (built Oct 12 2007; 32bit)) (this is one line of course) Is it possible that the problem about ')' is caused by this line? Or is it tottaly different issue? I just think this, because the the major difference between this and the working server on migration is to have ')' and '(' at the imap "greeting", the working server does not have those, and the error message is about something with ')'. The command was the following (no configuration in dovecot for this, only this command is given at shell level - currently mainly for testing if it works or not); doveadm -D -v -o imapc_features=rfc822.size \ -o imapc_host=192.168.10.100 \ -o imapc_user=migtest at office -o imapc_password=SECRET backup -R -f \ -u migtest at office imapc: This was OK with a dovecot v1 as the "source" (as I've written) but has the problem I've desribed with that "old" sun messaging server as the source. I've also used tcpdump then wireshark's "follow TCP stream" on it: the communication seems to be just that "greeting" line, then my (target) dovecot sends FIN. That "old" server works otherwise since years with various IMAP clients. The log of my server does not contain too much just the db lookups, which works nicely (the same for working and non-working source IMAP servers): Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: prefetch(migtest at office): passdb didn't return userdb entries, trying the next userdb Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): user search: base=cn=mail,dc=office,dc=intra scope=subtree filter=(&(objectClass=mailUser)(uid=migtest at office)) fields=uid Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest at office): result: uid=migtest at office Any help is welcome, - G?bor From ml at cibest.com Mon Sep 10 18:03:39 2012 From: ml at cibest.com (Cibest ML) Date: Mon, 10 Sep 2012 17:03:39 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <20120909213052.GA11444@daniel.localdomain> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <20120909213052.GA11444@daniel.localdomain> Message-ID: <504E014B.8040309@cibest.com> On 09/09/2012 23:30, Daniel Parthey wrote: > Cibest ML wrote: >> On 08/09/2012 01:38, Daniel Parthey wrote: >>> Cibest ML wrote: >>>> We've a strange quota behavior on our dovecot servers. >>>> It seems that it's somehow doubling the real used storage (or could >>>> be something like twice the inbox size + size of other folders, hard >>>> to know for sure). >>>> It was initial setup with the maildir++ system. Now it's been >>>> configured with dict and mysql table but the result is the same. >> Here are some tests to show the problem. The test mailbox is >> initialy empty. All described actions are done in a sequencial way >> (test mail with an attachement of ~2MB) >> >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems ok. > >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Strange. There must be some issue with quota recalculation. > Maybe there are two instances of the message in the maildir? > How large is the actual maildir measured by diskusage (du) > command and where are the messages stored in the filesystem? > >> - move to draft + compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems absolutely correct. Message is copied, > deleted and removed from the old directory. > >> - recalc >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > Seems after compacting the folders there is > only one instance of the message. > >> - move to subfolder "test" in inbox >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> - move message back to inbox >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > The message gets copied and deleted, > so there are 2 messages. > >> - compact >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - > After compaction the "copied+deleted" > message gets finally removed. > >> - recalc quota >> Quota name Type Value Limit >> User quota STORAGE 5306 524288 >> User quota MESSAGE 2 - > Very strange. The removed message "reappeared"? > Are you using some network filesystem > and possibly experience filesystem caching effects? > In this case a dovecot director would help. > > Regards > Daniel The storage is indeed done via NFS. There might be some caching effects but in that case, the behaviour should occur for all the test cases. But it seems only to happen if the mails are stored in INBOX, not in a subfolder of INBOX nor another IMAP folder. We've never been able to reproduce the issue if the email is not exactly in INBOX. We've added some options to check if it's a NFS issue (but it shouldn't be needed as indexes aren't on a NFS filer) : map_disable = yes mail_fsync = always mail_nfs_index = yes It doesn't change anything, same behavior. Here are some test cases to show that there's always 1 mail stored on the filesystem new incoming mail du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota get Quota name Type Value Limit User quota STORAGE 3 524288 User quota MESSAGE 1 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 quota recacl Quota name Type Value Limit User quota STORAGE 7 524288 User quota MESSAGE 2 - du -> 8 ls -> -rw------- 1 vmail vmail 3748 Sep 10 16:11 1347286291.M194531P4329.mspooll01,S=3748,W=3847 it really seems to be connected to the recalc operation. Moving to dovecot director would require some work but at the moment, we can't be sure it'd solve the issue (as we only have one IMAP/POP server + one LMTP server, we shouldn't have issues with stickiness of connexions) -- Regards S?bastien From roundcube222 at alaadin.org Mon Sep 10 20:28:37 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Mon, 10 Sep 2012 20:28:37 +0300 Subject: [Dovecot] Chown ChgrpNoPerm problem Message-ID: <3d5d646091b201e8545b990fc51c28b8@Coptics.org> Hello, I just have a one simple question. my logs has the following entries Sep 9 02:48:10 dovecot: pop3(r): Error: fchown(/home/r/mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) Sep 9 11:22:30 dovecot: pop3(r): Error: fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=501(r), group based on /var/mail/r) i know that chmod 0600 /var/mail , will solve the problem and i will no longer receive the above errors again But my question is that incase i did not set chmod /var/mail 0600, can i ignore such errors, is these errors harmful ? if this errors keep coming and i ignored them would this cause mbx corruption .. please advise My dovecot conf is disable_plaintext_auth = no mail_debug = yes mail_location = mbox:~/mymail:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } ssl_cert = hello timo and other dev after a precise reading of plugin deleted to trash it appears to me like things more varariables are declared NULL in the code. it appears to me as being possible to achieve actions even if messages with the same id appears several times in different INBOX when it is the exact order of things -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://about.me/fakessh http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From ben at indietorrent.org Mon Sep 10 22:19:05 2012 From: ben at indietorrent.org (Ben Johnson) Date: Mon, 10 Sep 2012 15:19:05 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <5048E3EA.80009@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> Message-ID: <504E3D29.10102@indietorrent.org> On 9/6/2012 1:56 PM, Ben Johnson wrote: > > > On 9/6/2012 6:10 AM, Charles Marcus wrote: >> On 2012-09-05 6:20 PM, Ben Johnson wrote: >> >>> My configuration is Dovecot (1.2.9) + Sieve + SpamAssassin on Ubuntu >> 10.04. >> >> 1.2.9 is really old... you really need to upgrade to a recent/stable >> version. > > Thanks, Charles. I do see your point. One of the challenges we face in > this regard is that we're using a Long-Term-Support version of Ubuntu > (10.04) and 1.2.9 is the latest package in the OS's repository. > > That said, we could upgrade manually, but this is a production server on > which downtime must be minimized, and we all know how unexpected issues > arise during installation (even when the procedure is tested in a > closely equivalent development environment). > >>> I followed the Wiki article at http://wiki2.dovecot.org/Plugins/Antispam >> >> That is for version 2.x (note the wiki2)... > > Noted; thank you. > >> I recommend going ahead and upgrading to the latest 2.1.9 and starting >> over. >> > > Maybe I'll wait until we upgrade from Ubuntu 10.04 to 12.04 to fiddle > with this further. > > Thanks again, > > -Ben > So, I setup a development environment that mimics the production environment in question. Must I install Dovecot from source to install Antispam from source? Because I installed Dovecot using a back-ported package ( https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and when I go to install the Antispam plug-in from source, executing the "configure" script yields the following error: # ~/dovecot-antispam-plugin# ./configure [...] checking for dovecot-config in "/usr/local/lib/dovecot"... not found configure: configure: Use --with-dovecot=DIR to provide the path to the dovecot-config file. configure: error: dovecot-config not found I'm assuming that this directory does not exist on my system because I didn't install Dovecot from source. Is this accurate? If so, is my only option to install Dovecot from source and then attempt the same with the Antispam plug-in? Thanks for any pointers, -Ben From silent_wumpus at yahoo.com Tue Sep 11 00:49:12 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Mon, 10 Sep 2012 14:49:12 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <1347313752.51294.YahooMailNeo@web121502.mail.ne1.yahoo.com> I have further read the Migration documentation and someone pointed out to me two good questions: 1 - If I use the modified mb2md.pl -U the program creates a dovecot-uidlist. Or if I can produce this list is there a way I can make dovecot use this list when it creates the dovecot-index file sets the first time it accesses the mailbox? Aka is there a command line option using dovecot or doveadm that creates the indexes based on the dovecot-uidlist that I can script for each mailbox I transfer? 2 - How/where/when do I use "pop3_reuse_xuidl=yes" in the dovecot.conf file for the first time each mailbox is read? Does this option need to be used after the first time (it is not set by default in my new server config). ________________________________ From: Tim E. To: Dovecot List Mailing Sent: Friday, September 7, 2012 3:32 PM Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done. From jtam.home at gmail.com Tue Sep 11 02:03:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 10 Sep 2012 16:03:46 -0700 (PDT) Subject: [Dovecot] ChgrpNoPerm issue In-Reply-To: References: Message-ID: Robert JR writes: > The problem is /var/mail (Owner is > useraccount and the group us mail) , and here comes the problem > Dovecot > keeps trying to chown the imap folder inside the homedirectory with > user:mail account and since > And this is the reason of error that appears > in my log file. Dovecot is trying to keep the permission of your index caches consistent with that of your mailbox it indexes. The INBOX index cache is kept in your user's mail directory (as per your default settings), although you can change that location. > The option mail_access_groups=mail solve the problem.. > but I read it is not secure.. > With my current configutation, users login > with imap to read mail , can they use > any the mail_access_groups=mail > and read other poeple mail ? Yes, that's the security problem. > does my configutation > is a shard mailbox > and could be unsecure.. Could not parse your question/comment. > why didnot have > this issue with uw-imapd? and why dovecot try to chown the .imap > folder > with the mail group ? uw-imapd was not as picky. The extra consideration for group ownership is so that shared access to mailbox files, and their associated index caches, remain consistent. For example, if you shared a mailbox among your colleagues in group "staff" and the mailbox has group=staff,perm=g+rw, then the index caches will inherit those permissions, and members of group staff can access mailbox and indices alike. [You later write ...] > Sep 9 11:22:30 dovecot: pop3(r): Error: > fchown(/home/r/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) > failed: Operation not permitted (egid=501(r), group based on > /var/mail/r) > > i know that chmod 0600 /var/mail , will solve the problem > and i will no longer receive the above errors again You also have to make sure that autocreated INBOXs (i.e. a brand new account) does not start out with anything other than 0600. You may have to use dovecot's LDA or twist your LDA's arm to create mailboxes that way. I guess you can also avoid these errors by using memory indices, but you forego the advantages of persistent indices. > But my question is > that incase i did not set chmod /var/mail 0600, can i ignore such > errors, is these errors harmful ? if this errors keep coming and i > ignored them would this cause mbx corruption .. please advise No, you can't ignore these errors. They will break IMAP access to those mailboxes (as you will find out). Joseph Tam From ben at morrow.me.uk Tue Sep 11 04:14:56 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Tue, 11 Sep 2012 02:14:56 +0100 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504E3D29.10102@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> Message-ID: <20120911011456.GA3908@anubis.morrow.me.uk> At 3PM -0400 on 10/09/12 Ben Johnson wrote: > > So, I setup a development environment that mimics the production > environment in question. > > Must I install Dovecot from source to install Antispam from source? > > Because I installed Dovecot using a back-ported package ( > https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and > when I go to install the Antispam plug-in from source, executing the > "configure" script yields the following error: > > # ~/dovecot-antispam-plugin# ./configure > [...] > checking for dovecot-config in "/usr/local/lib/dovecot"... not found > configure: > configure: Use --with-dovecot=DIR to provide the path to the > dovecot-config file. > configure: error: dovecot-config not found > > I'm assuming that this directory does not exist on my system because I > didn't install Dovecot from source. It looks to me like you need to install the dovecot-dev package from that site (if you haven't already), and pass --with-dovecot=/usr/lib/dovecot to configure. Ben From ramon.frontera at uib.es Tue Sep 11 12:41:04 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Tue, 11 Sep 2012 11:41:04 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers Message-ID: Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. We have 2 proxies with director and 4 mail servers. I have configured expire plugin on a single mail server and it works Ok. I don't konw how to configure with 4 mail servers and 2 proxies. My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command Do you have any idea how is the best way to configure expire plugin on this scenario? Thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears From robert at schetterer.org Tue Sep 11 12:58:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 11 Sep 2012 11:58:20 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <504F0B3C.1020105@schetterer.org> Am 11.09.2012 11:41, schrieb Ram?n Frontera: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command > Do you have any idea how is the best way to configure expire plugin on this scenario? > > Thanks in advance. > > -- > Ramon Frontera Gallardo > Universitat de les Illes Balears > > > > > i use one mysql setup for all servers http://wiki2.dovecot.org/Plugins/Expire and doing cron jobs on every server i.e doveadm expunge -A mailbox Trash savedbefore 30d fit the cron times and expunge times and folders to your needs prepare for cron mail messages what got expunged if you have lots off acounts there may be better ways -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Tue Sep 11 15:40:14 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 11 Sep 2012 14:40:14 +0200 Subject: [Dovecot] Is it possible to save only some messages compressed to mdbox? (2.1.6) In-Reply-To: References: <20120820130617.GT24203@ruhr-uni-bochum.de> Message-ID: <20120911124014.GU3002@ruhr-uni-bochum.de> On Thu, Aug 23, 2012 at 11:08:38PM +0300, Timo Sirainen wrote: > > On 20.8.2012, at 16.06, Jost Krieger wrote: > > > So here's the questions: > > > > 1. Is it possible to have compressed/uncompressed mails mixed in an > > mdbox? > > Yes. Compression is detected on a per-mail basis. > > > 2. Will it work via "-o" in lda delivery? > > > > I tried to find out in the source but found it hard to see which options > > are pulled in when. > > -o can override all settings that are handled by the running process itself, so yes that includes compression stuff. Thank you for all your help, but I just can't get it to work. What I have now: For some mailboxes I pt extra parameters in the delivery command (via maildrop): EXTRACOMPRESS="-o debug_log_path=/tmp/lda-debug-log -o mail_debug=yes -o plugin/zlib_save_level=9 -o plugin/zlib_save=bz2" These parameters do arrive in lda: 1. Logging works. 2. If I replace plugin/zlib_save_level=9 with plugin/zlib_save_level=10 I get the correct error message from the zlib plugin. There are no problems in the logs and the mails arrive safely. There is just no compressed material in my mdbox storage files. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From tss at iki.fi Tue Sep 11 16:34:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 16:34:50 +0300 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: On 11.9.2012, at 12.41, Ram?n Frontera wrote: > Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. > We have 2 proxies with director and 4 mail servers. > > I have configured expire plugin on a single mail server and it works Ok. > I don't konw how to configure with 4 mail servers and 2 proxies. > My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. From anmeyer at anup.de Tue Sep 11 17:00:38 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:00:38 +0200 Subject: [Dovecot] imap_idle_notify_interval Message-ID: <20120911160038.66c3a387@itx.bitcorner.intern> Hi! I want to change the imap_idle_notify_interval t0 29 minutes and have edited the 20-imap.conf with imap_idle_notify_interval = 29 mins and restarted the server. doveconf -n shows me this: .... protocol imap { imap_idle_notify_interval = 29 mins mail_max_userip_connections = 10 mail_plugins = " quota imap_quota autocreate" } .... doveconf |grep idle shows me the following output: default_idle_kill = 1 mins imap_idle_notify_interval = 2 mins mailbox_idle_check_interval = 30 secs idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 0 idle_kill = 4294967295 secs idle_kill = 0 idle_kill = 4294967295 secs imap_idle_notify_interval = 29 mins I see two line of imap_idle_notify_interval. What is the right one? Greetings Andreas From tss at iki.fi Tue Sep 11 17:02:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:02:54 +0300 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: <20120911160038.66c3a387@itx.bitcorner.intern> References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: On 11.9.2012, at 17.00, Andreas Meyer wrote: > protocol imap { > imap_idle_notify_interval = 29 mins > mail_max_userip_connections = 10 > mail_plugins = " quota imap_quota autocreate" > } > .... > > doveconf |grep idle shows me the following output: > imap_idle_notify_interval = 2 mins > imap_idle_notify_interval = 29 mins > > I see two line of imap_idle_notify_interval. What is the right one? One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. From anmeyer at anup.de Tue Sep 11 17:06:17 2012 From: anmeyer at anup.de (Andreas Meyer) Date: Tue, 11 Sep 2012 16:06:17 +0200 Subject: [Dovecot] imap_idle_notify_interval In-Reply-To: References: <20120911160038.66c3a387@itx.bitcorner.intern> Message-ID: <20120911160617.7bc18f58@itx.bitcorner.intern> Hi! Timo Sirainen wrote: > On 11.9.2012, at 17.00, Andreas Meyer wrote: > > > protocol imap { > > imap_idle_notify_interval = 29 mins > > mail_max_userip_connections = 10 > > mail_plugins = " quota imap_quota autocreate" > > } > > .... > > > > doveconf |grep idle shows me the following output: > > imap_idle_notify_interval = 2 mins > > imap_idle_notify_interval = 29 mins > > > > I see two line of imap_idle_notify_interval. What is the right one? > > One is used by IMAP processes, the other is used by non-IMAP processes. Of course, non-IMAP processes don't use that setting. So everthing is fine and the imap_idle_notify_interval = 29 mins Thank you! Andreas From tss at iki.fi Tue Sep 11 17:14:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:14:27 +0300 Subject: [Dovecot] ACL MySQL dict: to_user==from_user entries In-Reply-To: <20120829124447.GA16592@sys-241.netcologne.de> References: <20120829124447.GA16592@sys-241.netcologne.de> Message-ID: <217953AF-1EFC-45E3-B5F1-F1BF0938F4FC@iki.fi> On 29.8.2012, at 15.44, Roland Rosenfeld wrote: > But after "SETACL INBOX user at domain lrwstipekxacd" I see a new entry > in my imap_acl_shares table where to_user and from_user both contain > the value "user at domain". > > Does it make sense to run a daily cron job, that removes all entries > with from_user=to_user or may this cause other inconsistencies? Yes. Also fixed in hg: http://hg.dovecot.org/dovecot-2.1/rev/3eaeacc85b4c From tss at iki.fi Tue Sep 11 17:17:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:17:59 +0300 Subject: [Dovecot] Alerts when process limits are met In-Reply-To: <5041C2FD.1040108@wordshell.net> References: <5041C2FD.1040108@wordshell.net> Message-ID: On 1.9.2012, at 11.10, David Anderson wrote: > I've not found the answer to this question anywhere - please forgive me if I overlooked. > > I'd like to be able to be automatically alerted if process limits are hit (e.g. max POP3 logins). > > Is there a way that I can configure a script to be run, in the same way that I can with quota warnings? > > I can of course use logwatch, but this alerts me the next day, and logwatch can be noisy and it's easy to overlook. Log errors/warnings to a separate file and watch it. Dovecot's error/warning log typically stays completely empty (except when stopping Dovecot). Anything you find there is a potential error.. > My motivation: for some reason my POP3 listener was hanging, and I had to restart dovecot (this is 2.0.9). But I was not aware - the process was still there, it was accepting connections (but not doing anything useful with them) and was actually managing to log the fact that the process limit was being hit (as more and more connections came in). If I could have been alerted I could have restarted dovecot earlier. Restarting is kind of a kludgy solution, since the only thing it does is to kill all the existing connections and hope that they don't immediately just reconnect back. An equivalent fix is doveadm kick '*' From tss at iki.fi Tue Sep 11 17:24:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:24:46 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50449193.8080101@um.es> References: <50449193.8080101@um.es> Message-ID: On 3.9.2012, at 14.16, Angel L. Mateo wrote: > Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): > > doveadm auth cache flush > 4 cache entries flushed Yep. > So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) Works in my tests. From tss at iki.fi Tue Sep 11 17:31:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:31:16 +0300 Subject: [Dovecot] dovecot proxy ssl-parameter from database? In-Reply-To: <5044B187.6000609@dhag.net> References: <5044B187.6000609@dhag.net> Message-ID: On 3.9.2012, at 16.32, Daniel Funke wrote: > So far all works fine. My question is the TLS or noTLS connection from the client to the proxy. How can I make the ssl-parameter user-dependant? clients to the old courier server should work without or with tls through the proxy and clients to the new dovecot server should only work WITH TLS between Client and proxy. (After migration of all clients the proxy should be switched off an all clients connect directly to the dovecot server.) You can't. If disable_plaintext_auth=yes or ssl=required, Dovecot fails the login before any passdb or userdb lookup is done. From tss at iki.fi Tue Sep 11 17:33:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 17:33:10 +0300 Subject: [Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6) In-Reply-To: <20120903141121.GR3002@ruhr-uni-bochum.de> References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: On 3.9.2012, at 17.11, Jost Krieger wrote: >>> imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) >> >> This looks like it does. Existing emails must not be changed, IMAP >> protocol forbids it (and Dovecot internals don't support it either). >> The only safe way to drop attachments is to assign a new UID for the >> mail, which basically means saving a new message without attachments >> and deleting the old one. The Content-Length header doesn't really >> have anything to do with this error. > > Sorry if I was unclear. This is exactly what the plugin does. It builds > a new message, starting with the old message and leaving out the > unwanted attchment(s). Then it saves the new message, deletes the old > message and finally copies over the flags. > > I had a look at the handling of "Content-Length" in dovecot and had the > impression that a new header was added but the old was not removed (if > formally ok) and on reading the first header wins. But I'm not sure, of > course. Any Content-Length header sent by IMAP client or via LDA input should always get dropped (among some other headers) and replaced with the correct value. So if there's a bug, I doubt it has anything to do with that. But if you can easily reproduce this failure it would be nice to get it fixed.. From ben at indietorrent.org Tue Sep 11 17:40:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 10:40:29 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <20120911011456.GA3908@anubis.morrow.me.uk> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> Message-ID: <504F4D5D.8040207@indietorrent.org> On 9/10/2012 9:14 PM, Ben Morrow wrote: > At 3PM -0400 on 10/09/12 Ben Johnson wrote: >> >> So, I setup a development environment that mimics the production >> environment in question. >> >> Must I install Dovecot from source to install Antispam from source? >> >> Because I installed Dovecot using a back-ported package ( >> https://launchpad.net/~christian-roessner-net/+archive/dovecot ) and >> when I go to install the Antispam plug-in from source, executing the >> "configure" script yields the following error: >> >> # ~/dovecot-antispam-plugin# ./configure >> [...] >> checking for dovecot-config in "/usr/local/lib/dovecot"... not found >> configure: >> configure: Use --with-dovecot=DIR to provide the path to the >> dovecot-config file. >> configure: error: dovecot-config not found >> >> I'm assuming that this directory does not exist on my system because I >> didn't install Dovecot from source. > > It looks to me like you need to install the dovecot-dev package from > that site (if you haven't already), and pass > --with-dovecot=/usr/lib/dovecot to configure. > > Ben > > You nailed it, Ben. Thanks! I downloaded/installed the dovecot-dev package from the same maintainer and was able to build Antispam without issue. The hang-up now seems to be with my configuration. I'm seeing the following error in my email client whenever I attempt to move an email message from, say, Inbox to Junk folder: [SERVERBUG] BUG Unknown internal error. Nothing at all appears in syslog when this happens. I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam , to the letter, I believe. The relevant portions of my configuration are as follows: --------------------- protocol imap { mail_plugins = quota imap_quota antispam } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve antispam_backend = MAILTRAIN antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } --------------------- The permissions and execute bit are set in accordance with other executables in the same directory: # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh I would provide more log content, but the debug options seem to be gone in the forked, >= 2.0 version of this plug-in. Any ideas are very much appreciated. Thanks, -Ben From tss at iki.fi Tue Sep 11 19:02:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:02:33 +0300 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <504647B2.5060804@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> Message-ID: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> On 4.9.2012, at 21.25, Florian Zeitz wrote: > Hello everyone and Timo in particular, > > about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present > in Dovecot. > I had always disliked this a bit, because it replicates a lot of code. > This patch generalizes the HMAC function to take a hash_method struct as > parameter, and changes existing code which uses the "old" HMAC function > to use this new one. > > I'm not really sure this is actually a good idea, but I still felt I > should provide the code in case you would want to merge it upstream. It's otherwise good, but this isn't safe: + ctx->ctx = t_malloc(meth->context_size); + ctx->ctxo = t_malloc(meth->context_size); It assumes that the hmac_init() + hmac_final() is called close to each others. I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) From tss at iki.fi Tue Sep 11 19:07:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:07:49 +0300 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <50474093.3010306@um.es> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> Message-ID: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> On 5.9.2012, at 15.07, Angel L. Mateo wrote: > El 05/09/12 13:50, Timo Sirainen escribi?: >> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >> >>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>> >>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>> >>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >> >> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >> > Ok, I'll try this. > > But... is there any way to know how a readdir would be done? What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. From tss at iki.fi Tue Sep 11 19:15:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:15:37 +0300 Subject: [Dovecot] Memory leak by getting the virtual size of a IMAP folder In-Reply-To: <5048CD0F.9030809@securepoint.de> References: <50471311.2050002@securepoint.de> <5048CD0F.9030809@securepoint.de> Message-ID: <766E466E-3264-42EC-BD35-EEFAE1E2FE17@iki.fi> On 6.9.2012, at 19.19, Dafan Zhai wrote: > I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. > > Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Yes, committed to hg. From tss at iki.fi Tue Sep 11 19:19:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:19:35 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> Message-ID: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> On 7.9.2012, at 10.30, Weinreich, Nico wrote: > I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. > > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation This crash was fixed in v2.1.8 I think. > Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. From tss at iki.fi Tue Sep 11 19:20:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:20:02 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049C861.2030904@um.es> References: <5049C861.2030904@um.es> Message-ID: On 7.9.2012, at 13.11, Angel L. Mateo wrote: > Hello, > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > What do you think is the best to activate it? Use dict-file quota instead of Maildir++ quota. From tss at iki.fi Tue Sep 11 19:23:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:23:22 +0300 Subject: [Dovecot] dovecot/log not running In-Reply-To: <5049DCEC.5050008@hinchfamily.com> References: <5049DCEC.5050008@hinchfamily.com> Message-ID: Something's very wrong. I have no idea what. doveconf -n output might help. On 7.9.2012, at 14.39, Mark Hinch wrote: > I'm running 2.1.7 as downloaded/installed by Debian (and > then appropriately setup for my config, ...). It almost > works. I can send/receive email from thunderbird, squirrelmail, ... > However there are two anomalies: > 1. dovecot/log is not running. After restarting dovecot (and waiting > 30 seconds), I have this in the log files: > > Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process > Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 > > After 30 seconds, a "ps -ef | grep dovecot" yields: > > dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login > root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w > mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap > root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot > root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login > dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil > root 25148 20223 0 18:55 ? 00:00:00 dovecot/config > dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth > root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params > mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap > > Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). > A "doveadm log find" dumps this in the log: > Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) > Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) > > However, further debugging that I try to do (see next problem) does not result in ANY > further input to the log files, even though I have all known logging turned on. So I *think* > that logging is not actually running. If I can get that started, then I'll take a crack at the > next problem myself ... > > 2. However, as long as I have your attention, my next problem is that it > seems like dovecot/imap-login hangs(?). After a few hours, I have something like: > > # ps -ef | grep dovecot > dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth > root 13277 31186 0 17:47 ? 00:00:00 dovecot/config > dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login > root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot > dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login > dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login > root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf > dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil > > with exactly 100 imap-login processes. I'm the only meaningful user > on the machine (unless somebody is imap DoS-ing me), so this does not > make much sense. Obviously thunderbird is polling the machine and each > time is leaving around a imap-log process. As far as I can tell, thunderbird > successfully logs in (i.e., I get mail) though I can't be sure whether these > are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? > After a while of course, dovecot will not start any new imap-login processes > so Thunderbird cannot get any more mail until I restart > dovecot. Then everything is OK for another 12 hours or so. > > Ideas on either problem? > > -Mark > From Ralf.Hildebrandt at charite.de Tue Sep 11 19:23:30 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:23:30 +0200 Subject: [Dovecot] FTS indexing Message-ID: <20120911162330.GA25501@charite.de> I'm reacreating all FTS indices on all my users' INBOX folders: mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes doveadm index -u %s -q INBOX In "ps auxwww" I'm seeing this: tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] how can a dovecot/indexer-worker actually access the INBOX of the user "tschuetz" (where "tbudkevy" doesn't have access, due to the different UID)...? Should ~%s/Maildir/lucene-indexes be filled after having run the command sequence for all users? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 19:25:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:25:24 +0300 Subject: [Dovecot] Quota less mailbox with mdbox In-Reply-To: References: Message-ID: <6C88B82F-A2FB-4644-AD0A-CB05A4E6C0ED@iki.fi> On 7.9.2012, at 13.43, H?ctor Moreno Blanco wrote: > Currently we have dovecot 2.0.10 with Maildir as mailbox system. We want to migrate to mdbox. > > The migration is simple and easy, but there are some configuration settings we cannot set to work properly. > > In our dovecot with Maildir we have the namespace INBOX with certain quota and another one without quota. The only difference should be that you need to switch from Maildir++ quota to dict quota backend. http://wiki2.dovecot.org/Quota/Dict From alec at alec.pl Tue Sep 11 19:29:59 2012 From: alec at alec.pl (A.L.E.C) Date: Tue, 11 Sep 2012 18:29:59 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> Message-ID: <504F6707.1030903@alec.pl> On 09/11/2012 06:19 PM, Timo Sirainen wrote: > The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. This is surprising. Can you provide more info about this issue? -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ben at indietorrent.org Tue Sep 11 19:34:29 2012 From: ben at indietorrent.org (Ben Johnson) Date: Tue, 11 Sep 2012 12:34:29 -0400 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <504F6815.90407@indietorrent.org> On 9/11/2012 12:23 PM, Timo Sirainen wrote: > Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> >> Not sure on the first issue. On the second issue, out of curiosity, what do you have defined in Thunderbird's Account Settings -> [account in question] -> Server Settings -> Advanced -> Maximum number of server connections to cache? Is this value actually set to 100? -Ben From tss at iki.fi Tue Sep 11 19:38:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 19:38:17 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911162330.GA25501@charite.de> References: <20120911162330.GA25501@charite.de> Message-ID: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > I'm reacreating all FTS indices on all my users' INBOX folders: > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > how can a dovecot/indexer-worker actually access the INBOX of the > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > different UID)...? indexer-worker runs basically as chroot, but it doesn't bother switching back to root immediately. My guess is that this was a very brief race condition in ps between reading process's UID and the command line. Unless you can easily reproduce this? > Should ~%s/Maildir/lucene-indexes be filled after having run the command > sequence for all users? Only if there was something in the INBOX. Also if users already had previous Lucene indexes it doesn't work, you need to run doveadm fts rescan because dovecot.index remembers which message was last indexed to Lucene. From Ralf.Hildebrandt at charite.de Tue Sep 11 19:54:07 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 18:54:07 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> Message-ID: <20120911165407.GD25501@charite.de> * Timo Sirainen : > On 11.9.2012, at 19.23, Ralf Hildebrandt wrote: > > > I'm reacreating all FTS indices on all my users' INBOX folders: > > mkdir ~%s/Maildir/lucene-indexes && chown %s:users ~%s/Maildir/lucene-indexes > > You shouldn't need to do this. It's created automatically. Also by "recreating" do you mean that the users already had Lucene indexes and you deleted them? There weren't any. That's why I tried creating them! > > tbudkevy 29186 3.7 0.0 6616 3056 ? S 18:11 0:15 dovecot/indexer-worker [tschuetz INBOX] > > > > how can a dovecot/indexer-worker actually access the INBOX of the > > user "tschuetz" (where "tbudkevy" doesn't have access, due to the > > different UID)...? > > indexer-worker runs basically as chroot, but it doesn't bother > switching back to root immediately. My guess is that this was a very > brief race condition in ps between reading process's UID and the > command line. Unless you can easily reproduce this? Can't :) > > Should ~%s/Maildir/lucene-indexes be filled after having run the command > > sequence for all users? > > Only if there was something in the INBOX. My users ALWAYS have something in their inbox :) > Also if users already had previous Lucene indexes it doesn't work, you > need to run doveadm fts rescan because dovecot.index remembers which > message was last indexed to Lucene. OK; I'll try that. Where do I finde the lucene index? I didn't find any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to create the directories. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Tue Sep 11 20:04:32 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:04:32 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911165407.GD25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> Message-ID: <20120911170432.GE25501@charite.de> * Ralf Hildebrandt : > OK; I'll try that. Where do I finde the lucene index? I didn't find > any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to > create the directories. I fucked up royally: I somehow managed to have fts but not fts_lucene in my list of plugins... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 11 20:47:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:47:23 +0300 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <20120910113307.GG14437@vega.lgb.hu> References: <20120910113307.GG14437@vega.lgb.hu> Message-ID: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> On 10.9.2012, at 14.33, G?bor L?n?rt wrote: > dsync(migtest at office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' > * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS > CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO > X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] > imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging > Server 6.3-5.02 (built Oct 12 2007; 32bit)) > > Is it possible that the problem about ')' is caused by this line? I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 From tss at iki.fi Tue Sep 11 20:49:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 20:49:04 +0300 Subject: [Dovecot] FTS indexing In-Reply-To: <20120911170432.GE25501@charite.de> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> Message-ID: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> On 11.9.2012, at 20.04, Ralf Hildebrandt wrote: > * Ralf Hildebrandt : > >> OK; I'll try that. Where do I finde the lucene index? I didn't find >> any trace of ~%s/Maildir/lucene-indexes so I was worried and tried to >> create the directories. > > I fucked up royally: I somehow managed to have fts but not fts_lucene > in my list of plugins... It should log errors about that if plugin { fts=lucene } I think? From Ralf.Hildebrandt at charite.de Tue Sep 11 20:51:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 11 Sep 2012 19:51:45 +0200 Subject: [Dovecot] FTS indexing In-Reply-To: <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> References: <20120911162330.GA25501@charite.de> <9B01F7DA-A770-494F-90F8-13D0D6EA3AE0@iki.fi> <20120911165407.GD25501@charite.de> <20120911170432.GE25501@charite.de> <71E9C0B1-DEF9-4F98-9B4E-E595FED540C7@iki.fi> Message-ID: <20120911175145.GJ25501@charite.de> * Timo Sirainen : > > I fucked up royally: I somehow managed to have fts but not fts_lucene > > in my list of plugins... > > It should log errors about that if plugin { fts=lucene } I think? Well, I somehow managed to sprinkle the plugin lines in such a way that no error was triggered. Now I put fts and fts_lucene into mail_plugins and everything works. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From florob at babelmonkeys.de Tue Sep 11 21:07:05 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 11 Sep 2012 20:07:05 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> Message-ID: <504F7DC9.3040408@babelmonkeys.de> Am 11.09.2012 18:02, schrieb Timo Sirainen: > On 4.9.2012, at 21.25, Florian Zeitz wrote: > >> Hello everyone and Timo in particular, >> >> about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present >> in Dovecot. >> I had always disliked this a bit, because it replicates a lot of code. >> This patch generalizes the HMAC function to take a hash_method struct as >> parameter, and changes existing code which uses the "old" HMAC function >> to use this new one. >> >> I'm not really sure this is actually a good idea, but I still felt I >> should provide the code in case you would want to merge it upstream. > > It's otherwise good, but this isn't safe: > > + ctx->ctx = t_malloc(meth->context_size); > + ctx->ctxo = t_malloc(meth->context_size); > > It assumes that the hmac_init() + hmac_final() is called close to each others. I had in fact noticed that. The assumption is currently true for all occurrences, and probably will remain such, but I agree it's better to be safe then sorry. > I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) > Well, either that, or we could use a union of all known context structs there. Possibly plus an i_assert(meth->context_size <= sizeof(union hmac_ctxts)). Or we could use i_malloc() and i_free() under the assumption hmac_init() + hmac_final() calls are always matched. I've a certain preference for the union variant, but it's your call. Regards, Florian From tss at iki.fi Tue Sep 11 21:43:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:43:34 +0300 Subject: [Dovecot] v2.1 memory usage Message-ID: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes? ./configure --with-storages=mdbox sdbox maildir mbox cydir I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :) Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something.. From tss at iki.fi Tue Sep 11 21:48:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:48:08 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <504B5F2E.7070202@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> Message-ID: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> On 8.9.2012, at 18.07, Cibest ML wrote: > - Message delivered by LMTP > Quota name Type Value Limit > User quota STORAGE 2653 524288 > User quota MESSAGE 1 - > > - quota recalc > Quota name Type Value Limit > User quota STORAGE 5307 524288 > User quota MESSAGE 2 - Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? From tss at iki.fi Tue Sep 11 21:50:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 21:50:02 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> Message-ID: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> On 7.9.2012, at 22.32, Tim E. wrote: > I have a box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Tue Sep 11 23:31:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 11 Sep 2012 23:31:53 +0300 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <504F6707.1030903@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int> <9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> On 11.9.2012, at 19.29, A.L.E.C wrote: > On 09/11/2012 06:19 PM, Timo Sirainen wrote: > >> The error is that Roundcube doesn't send the string to Dovecot as UTF8 (or alternatively specify the correct charset). Even with the crashfix the search won't work. > > This is surprising. Can you provide more info about this issue? No idea if that's a misconfiguration or what .. or maybe the web browser says it's sending UTF8 but isn't. Anyway, with v2.1.7 it crashes if you send: a SEARCH subject "8bit non-utf8 text" I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation From lgb at lgb.hu Wed Sep 12 00:14:26 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Tue, 11 Sep 2012 23:14:26 +0200 Subject: [Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'" In-Reply-To: <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> References: <20120910113307.GG14437@vega.lgb.hu> <85B5BB55-EA57-4EA1-A04F-24F8588F194D@iki.fi> Message-ID: <20120911211426.GA15502@vega.lgb.hu> Hi, On Tue, Sep 11, 2012 at 08:47:23PM +0300, Timo Sirainen wrote: [...] > > Is it possible that the problem about ')' is caused by this line? > > I'm not sure if it's because of that line, but there are ways to make it break with that error message. These fix it: I just guessed it was, as the error msg was "server sent invalid input" but the only input from the server according to tcpdump was that line. For sure it's only my oversimplified logic now :) > http://hg.dovecot.org/dovecot-2.1/rev/382df961f290 > http://hg.dovecot.org/dovecot-2.1/rev/245fe7fd6f00 Thanks, Timo! With these patches the problem does not show up (honestly, just a quick test after patching/rebuilding, but it seems work now). Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Now I got this, but imho this is totally different issue (mine, not dovcot's) and I will take care about that later. Thanks for your help again! From tss at iki.fi Wed Sep 12 00:29:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 00:29:13 +0300 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <5044F66F.3010505@corp.sonic.net> References: <5044F66F.3010505@corp.sonic.net> Message-ID: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> On 3.9.2012, at 21.26, Kelsey Cummings wrote: > >Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected > >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected > >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected > >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected All of these connections had finished handshaking. They simply just seemed to disconnect the connection for no reason. I found one reason for why that could happen, which could explain all of this: http://hg.dovecot.org/dovecot-2.1/rev/24e791bbcf69 The user weakness is remembered until all directors are shut down or all directors have acknowledged the weakness. > >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left The user weakness should have been removed at this point, but looks like there's code missing for it for 1-director-only setup. I'll fix that soon.. > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) > >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) These are the weak users causing the trouble. This improves logging: http://hg.dovecot.org/dovecot-2.1/rev/27d3289e1f5c From lgb at lgb.hu Wed Sep 12 01:21:03 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 00:21:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: References: <5049C861.2030904@um.es> Message-ID: <20120911222103.GB15502@vega.lgb.hu> On Tue, Sep 11, 2012 at 07:20:02PM +0300, Timo Sirainen wrote: > On 7.9.2012, at 13.11, Angel L. Mateo wrote: > > > Hello, > > > > I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). > > > > I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). > > > > What do you think is the best to activate it? > > Use dict-file quota instead of Maildir++ quota. > Sorry to jump into this topic, but can I ask why? Is dict quota is superior over Maildir++ quota in performance in general or it is "only" about the fact, that it's better in the case like above: activating the quota for many users later than the creation of the server itself? The question is interesting for me as well, as I need to implement quota with maildir (on NFS). From tss at iki.fi Wed Sep 12 01:28:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:28:58 +0300 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120911222103.GB15502@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> Message-ID: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> On 12.9.2012, at 1.21, G?bor L?n?rt wrote: >>> I'm planning to activate quota control in dovecot, with maildir quota backend. I have about 70k users in my system directed to 4 backend servers (with a director to ensure that a user is always directed to the same server). >>> >>> I have tried to activate it in one of my nodes. The problem is that load of it has increase a lot, as much as the system was unusable (maildir is in nfs storage, with indexes in local disks). >>> >>> What do you think is the best to activate it? >> >> Use dict-file quota instead of Maildir++ quota. >> > > Sorry to jump into this topic, but can I ask why? Is dict quota is superior > over Maildir++ quota in performance in general or it is "only" about the > fact, that it's better in the case like above: activating the quota for many > users later than the creation of the server itself? Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. From silent_wumpus at yahoo.com Wed Sep 12 01:39:00 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Tue, 11 Sep 2012 15:39:00 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> Message-ID: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: dsync -v -u user backup user at domain.com All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. I ask this question since the mbox file does not have the associated dovecot indexes. I know Im foobaring/newbying something simple at this point. Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. ________________________________ From: Timo Sirainen To: Tim E. Cc: Dovecot List Mailing Sent: Tuesday, September 11, 2012 2:50 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 7.9.2012, at 22.32, Tim E. wrote: > I have a? box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support > XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir > but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: > > dsync : dovecot 1.x server is too old and doesnt support the command? > mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files > virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. > > Any thoughts would be appreciated. > I would like to produce a howto of the process when I am done. Copy the mboxes to the new v2.x server and use dsync to convert them to maildir. From tss at iki.fi Wed Sep 12 01:48:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 12 Sep 2012 01:48:53 +0300 Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> Message-ID: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From dmalolepszy at optusnet.com.au Wed Sep 12 08:31:32 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Wed, 12 Sep 2012 15:31:32 +1000 Subject: [Dovecot] Strip backslash from usernames Message-ID: <50501E34.2030900@optusnet.com.au> Hi, Is there a way to strip a backslash if it exists at the start of username like \user at example.com within the Dovecot LDA? I am using Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL trickery. Dominic. From alessio at skye.it Wed Sep 12 09:48:19 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 12 Sep 2012 08:48:19 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50503033.9010809@skye.it> Il 12/09/2012 00:28, Timo Sirainen ha scritto: > On 12.9.2012, at 1.21, G?bor L?n?rt wrote: > > >> Sorry to jump into this topic, but can I ask why? Is dict quota is superior >> over Maildir++ quota in performance in general or it is "only" about the >> fact, that it's better in the case like above: activating the quota for many >> users later than the creation of the server itself? > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > But dict quota, for enforce limits, can only be used with dovect as LDA? I'm using Maildir++ with qmail/vpopmail for local delivery and dovecot cannot enforce limits on delivery. With this configuration the only way to update dictionary is "doveadm -A quota rescan". Is correct? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From amateo at um.es Wed Sep 12 10:18:01 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 09:18:01 +0200 Subject: [Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning took seconds) In-Reply-To: <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> References: <50473A46.1080807@um.es> <32801DB0-0783-47B2-B990-83E791FA2899@iki.fi> <50474093.3010306@um.es> <7C41BF4B-325E-4A35-B904-97C1E5A91170@iki.fi> Message-ID: <50503729.1030502@um.es> El 11/09/12 18:07, Timo Sirainen escribi?: > On 5.9.2012, at 15.07, Angel L. Mateo wrote: > >> El 05/09/12 13:50, Timo Sirainen escribi?: >>> On 5.9.2012, at 14.40, Angel L. Mateo wrote: >>> >>>> Sep 3 12:35:49 myotis32 dovecot: imap(): Warning: Maildir: Scanning took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) >>>> >>>> We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). >>>> >>>> So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. >>> >>> That's the way Maildir works. If Dovecot is the only one accessing the maildir, you can set maildir_very_dirty_syncs=yes to reduce how often readdir()s are done (but it doesn't completely eliminate them in all situations). >>> >> Ok, I'll try this. >> >> But... is there any way to know how a readdir would be done? > > What do you mean by "how"? Without that setting readdir() is called any time the cur/ directory's mtime changes. With that setting it's done only when the mtime isn't what Dovecot expected it to be (saved in index). Also it's done if Dovecot can't open maildir file by guessing its name based on the flags in index file, which can happen if multiple clients access the mailbox at the same time. > I meant "when", not how, sorry. But my question is answer with this information. From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:24:28 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:24:28 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi> <504F6707.1030903@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From Nico.Weinreich at drefa-msg.de Wed Sep 12 10:28:51 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 09:28:51 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <99B44966-D340-423C-8A3E-6AEE49259B83@iki.fi> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0B@s-mcl-mx1.mediacity.drefa.int> Oh sorry, this seems to be an copy&paste failure. "dsr_sub_allocation" was one of the search terms (we run a data warehouse here) ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Timo Sirainen Gesendet: Di 11.09.2012 22:31 An: A.L.E.C Cc: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars [...] I tested a few other possibilities, but that was the only way I could make it crash. v2.1.8 fixes that crash. The only strange thing in the original bug report was the "dsr_sub_allocation" text: > Sep 5 11:43:32 mail dovecot: imap(abc at def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation -------------- next part -------------- A non-text attachment was scrubbed... Name: winmail.dat Type: application/ms-tnef Size: 3837 bytes Desc: not available URL: From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:04:48 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:04:48 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." -searching for a non-UTF-8 term and no mail is found then Roundcube shows correctly a not-found error Until here all is exactly the same, except, that I cannot see the Dovecot crash in /var/log/mail.log anymore. May I assume, that this is now a problem of Roundcube? (But if this is related to Roundcube, why do I still get an connection error (see above)? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von Weinreich, Nico Gesendet: Mi 12.09.2012 09:24 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars Yes, Roundcube had a similiar bug in past. But as I've seen in source code (of Roundcube), they encode the search string as UTF8. I'll give 2.1.8 a try and give feedback. Thanks so far From alec at alec.pl Wed Sep 12 11:16:06 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:16:06 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505044C6.7010504@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: > > -searching for a term containg UTF-8 chars and a mail is found in actual folder then all is fine > -searching for an UTF-8 term and no mail is found then Roundcube shows still "Connection to storage server failed." I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Nico.Weinreich at drefa-msg.de Wed Sep 12 11:31:08 2012 From: Nico.Weinreich at drefa-msg.de (Weinreich, Nico) Date: Wed, 12 Sep 2012 10:31:08 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl><01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int><01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505044C6.7010504@alec.pl> Message-ID: <01A2A7EAB4754943990A452783568F7007756E0E@s-mcl-mx1.mediacity.drefa.int> Yeah, this fixed it. Will this patch go into source? ________________________________ Von: dovecot-bounces at dovecot.org im Auftrag von A.L.E.C Gesendet: Mi 12.09.2012 10:16 An: dovecot at dovecot.org Betreff: Re: [Dovecot] Found bug on doing IMAP search with UTF8 chars I think this patch fixes the issue. https://github.com/roundcube/roundcubemail/commit/c093dcc391ad886ab815e6e70630c8e252df06ab ps. Roundcube has a nice imap_debug option, use it for debugging. From alec at alec.pl Wed Sep 12 11:37:30 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 10:37:30 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> Message-ID: <505049CA.6090202@alec.pl> On 09/12/2012 10:04 AM, Weinreich, Nico wrote: > I've installed 2.1.9 from source (and Pigeonhole also). The error in Roundcube is still the same: I've found what's really going on here. This is initial search request with utf-8: C: A0003 UID SORT (DATE) UTF-8 ALL OR HEADER SUBJECT {10+} C: L?sung123 HEADER FROM {10+} C: L?sung123 S: * SORT S: A0003 OK Sort completed. Now, because of a bug, fixed in commit I posted before, Roundcube sends another request with search arguments converted to US-ASCII. It should do this only if the first command returns "unsupported charset" response. C: A0004 UID SORT (DATE) US-ASCII ALL OR HEADER SUBJECT {9+} C: L.sung123 HEADER FROM {9+} C: L.sung123 S: A0004 NO Invalid search key The problem is that on that convertion ? is converted to character with code 246 (hex F6), which is from extended ASCII table. This is because in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix this in Roundcube. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From ramon.frontera at uib.es Wed Sep 12 11:53:37 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 10:53:37 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure Message-ID: Hello we are running Dovecot 2.1.9. We have 2 proxies with director and 4 mail servers. We have the user information in a LDAP. I try to do: # doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed user mail server ip expire time x.x.x.x 2012-09-12 10:52:17 x.x.x.x 2012-09-12 10:55:53 in the dovecot log i have found: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it What can I do to solve this problem? Thanks in advance. -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears From ml at cibest.com Wed Sep 12 11:58:59 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 12 Sep 2012 10:58:59 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> Message-ID: <50504ED3.5030601@cibest.com> On 11/09/2012 20:48, Timo Sirainen wrote: > On 8.9.2012, at 18.07, Cibest ML wrote: > >> - Message delivered by LMTP >> Quota name Type Value Limit >> User quota STORAGE 2653 524288 >> User quota MESSAGE 1 - >> >> - quota recalc >> Quota name Type Value Limit >> User quota STORAGE 5307 524288 >> User quota MESSAGE 2 - > Quota recalculation is clearly seeing the message twice. My first guess was that you had two namespaces pointing to the same location, but you seem to have only one. Do you have symlinks in your Maildir? > Hi, Nothing specific on the namespace side. The only entry is the default one in 10-mail.conf namespace inbox { inbox = yes } I tried to comment it out but it doesn't change anything And no symlinks in the Maildir. Here's the file structure of Maildir for the test mailbox ./dovecot-uidvalidity.504df4e7 ./dovecot-uidvalidity ./.Sent ./.Sent/maildirfolder ./.Sent/new ./.Sent/dovecot.index.log ./.Sent/tmp ./.Sent/dovecot-uidlist ./.Sent/cur ./.Spam ./.Spam/maildirfolder ./.Spam/new ./.Spam/dovecot.index.log ./.Spam/tmp ./.Spam/dovecot-uidlist ./.Spam/cur ./new ./subscriptions ./.Drafts ./.Drafts/maildirfolder ./.Drafts/new ./.Drafts/dovecot.index.log ./.Drafts/tmp ./.Drafts/dovecot-uidlist ./.Drafts/cur ./.Drafts/dovecot-keywords ./dovecot.index.log ./tmp ./dovecot.mailbox.log ./dovecot-uidlist ./.Trash ./.Trash/maildirfolder ./.Trash/new ./.Trash/dovecot.index.log ./.Trash/tmp ./.Trash/dovecot-uidlist ./.Trash/cur ./cur ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa ./dovecot-keywords ./dovecot.index.cache -- Regards S?bastien From alec at alec.pl Wed Sep 12 12:22:27 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 12 Sep 2012 11:22:27 +0200 Subject: [Dovecot] Found bug on doing IMAP search with UTF8 chars In-Reply-To: <505049CA.6090202@alec.pl> References: <01A2A7EAB4754943990A452783568F7007756E09@s-mcl-mx1.mediacity.drefa.int><9FCCD73E-69ED-4A3A-BFAC-6C6743C5CA18@iki.fi><504F6707.1030903@alec.pl> <01A2A7EAB4754943990A452783568F7007756E0A@s-mcl-mx1.mediacity.drefa.int> <01A2A7EAB4754943990A452783568F7007756E0C@s-mcl-mx1.mediacity.drefa.int> <505049CA.6090202@alec.pl> Message-ID: <50505453.4050603@alec.pl> On 09/12/2012 10:37 AM, A.L.E.C wrote: > The problem is that on that convertion ? is converted to character with > code 246 (hex F6), which is from extended ASCII table. This is because > in PHP (iconv/mb_string) US-ASCII means extended ASCII. So, I'll fix > this in Roundcube. Euh. Of course not PHP issue, the bug was in another place. Fixed. https://github.com/roundcube/roundcubemail/commit/40e1c2d14d11c679d97c00ce831b32d43fad4d32 The fixes are applied to master and release-0.8 branches. ps. sorry for spamming your list, I think it's EOT now. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From amateo at um.es Wed Sep 12 12:54:29 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 11:54:29 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <50505BD5.70907@um.es> El 12/09/12 00:28, Timo Sirainen escribi?: > > Activating quota is equally slow, because it needs to scan sizes of all files. If the size is stored in maildir filename (,S=123) this is pretty fast, otherwise it's pretty slow because Dovecot stat()s the files, but afterwards it stores them to dovecot-uidlist file. So the main difference is that Maildir++ requires rescanning the quota periodically, while dict quota never rescans unless you run doveadm quota rescan. > doveadm quota rescan (not documented)? or doveadm quota recalc (as documented)? Anyway, I'll use dict instead of maildir backend. From mhinch at hinchfamily.com Wed Sep 12 14:13:55 2012 From: mhinch at hinchfamily.com (Mark Hinch) Date: Wed, 12 Sep 2012 06:13:55 -0500 Subject: [Dovecot] dovecot/log not running In-Reply-To: References: <5049DCEC.5050008@hinchfamily.com> Message-ID: <50506E73.30609@hinchfamily.com> Answers for both my helpers - Timo and Ben (Thanks!): Timo: Problem #1 - no dovecot/login doveconf -n output: ============== # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.30-2-686 i686 Debian wheezy/sid mail_access_groups = mail mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = " imap" ssl_cert = Something's very wrong. I have no idea what. doveconf -n output might help. > > On 7.9.2012, at 14.39, Mark Hinch wrote: > >> I'm running 2.1.7 as downloaded/installed by Debian (and >> then appropriately setup for my config, ...). It almost >> works. I can send/receive email from thunderbird, squirrelmail, ... >> However there are two anomalies: >> 1. dovecot/log is not running. After restarting dovecot (and waiting >> 30 seconds), I have this in the log files: >> >> Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process >> Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 >> >> After 30 seconds, a "ps -ef | grep dovecot" yields: >> >> dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login >> root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w >> mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap >> root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot >> root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login >> dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil >> root 25148 20223 0 18:55 ? 00:00:00 dovecot/config >> dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth >> root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params >> mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap >> >> Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). >> A "doveadm log find" dumps this in the log: >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) >> Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) >> >> However, further debugging that I try to do (see next problem) does not result in ANY >> further input to the log files, even though I have all known logging turned on. So I *think* >> that logging is not actually running. If I can get that started, then I'll take a crack at the >> next problem myself ... >> >> 2. However, as long as I have your attention, my next problem is that it >> seems like dovecot/imap-login hangs(?). After a few hours, I have something like: >> >> # ps -ef | grep dovecot >> dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth >> root 13277 31186 0 17:47 ? 00:00:00 dovecot/config >> dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login >> root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot >> dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login >> root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf >> dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil >> >> with exactly 100 imap-login processes. I'm the only meaningful user >> on the machine (unless somebody is imap DoS-ing me), so this does not >> make much sense. Obviously thunderbird is polling the machine and each >> time is leaving around a imap-log process. As far as I can tell, thunderbird >> successfully logs in (i.e., I get mail) though I can't be sure whether these >> are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? >> After a while of course, dovecot will not start any new imap-login processes >> so Thunderbird cannot get any more mail until I restart >> dovecot. Then everything is OK for another 12 hours or so. >> >> Ideas on either problem? >> >> -Mark >> From lgb at lgb.hu Wed Sep 12 14:34:07 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 13:34:07 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> Message-ID: <20120912113407.GG22070@vega.lgb.hu> On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: > Activating quota is equally slow, because it needs to scan sizes of all > files. If the size is stored in maildir filename (,S=123) this is pretty > fast, otherwise it's pretty slow because Dovecot stat()s the files, but > afterwards it stores them to dovecot-uidlist file. So the main difference > is that Maildir++ requires rescanning the quota periodically, while dict > quota never rescans unless you run doveadm quota rescan. Ahaa, thanks for your answer. But ... why is rescan needed sometimes with maildir++? If I don't touch maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, not even the LDA, etc), then why is it needed to rescan periodically with maildir? I don't really understand the difference then which needs quota rescan with maildir++ but not with dict quota during the very same usage environment otherwise. Anyway I guess, having dict quota with SQL backend is not so a bad idea: I would be able to check incoming mails (at MXs) if target user will have enough space to hold the mail by using the mail size and quota usage (from SQL what dict quota uses as well) on the MXs using eg postfix policy server ... So I won't generate ugly NDRs later, after accepted mails. Not 100% accurate (quota usage may change while mail hits dovecot) but at least it will stop the majority of those quota related NDRs I have problems with currently. The only thing which keep me away from dict quota with sql backend that it needs much more complex stuff than simply using maildir++ quota, but it's maybe only my lazyness ... thanks again. From ramon.frontera at uib.es Wed Sep 12 16:44:27 2012 From: ramon.frontera at uib.es (=?iso-8859-1?Q?Ram=F3n_Frontera?=) Date: Wed, 12 Sep 2012 15:44:27 +0200 Subject: [Dovecot] Expire plugin with multiple mail servers In-Reply-To: References: Message-ID: <61E579FF-DAB8-4569-BA10-8D599FBD0CA8@uib.es> Hello, I have configured mysql on one server and each mail server use this database for expire plugin. I have configured each mail server as explains on:http://wiki2.dovecot.org/Plugins/Expire But now I have a question; How i configure director server to use the database? I need to configure expire plugin on the director server? thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears El 11/09/2012, a las 15:34, Timo Sirainen escribi?: > On 11.9.2012, at 12.41, Ram?n Frontera wrote: > >> Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. >> We have 2 proxies with director and 4 mail servers. >> >> I have configured expire plugin on a single mail server and it works Ok. >> I don't konw how to configure with 4 mail servers and 2 proxies. >> My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. > > The backend servers need to be the ones actually doing the expunges. Director servers are the only ones that know which backend server to run each expunge on. Expire plugin works by limiting the users iterated by "doveadm expunge -A", which is run on director. So the director server where you run this command needs to have access to the full expire database. This pretty much means one shared SQL server between all servers. > From zinovik.igor at gmail.com Wed Sep 12 16:52:11 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Wed, 12 Sep 2012 17:52:11 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes Message-ID: Hello, list. I'm asking question not directly connected with dovecot. I'm planning a migration from cyrus-imapd to dovecot and i'm using cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes of users, but i'm wondering why size of mailbox in Maildir++ is so much bigger than mailbox in cyrus format after conversion: linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik /srv/vmail/petrsu.ru/z/zinovik/Maildir 238M /mnt/imap/z/user/zinovik 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir I was planning to implement quota for mailboxes about 1 gigabyte, but after conversion I would not be able to receive messages to my own box, because i'm overquota. I think the only way would be to set quota up to 15 GB since some of users have mailbox with 12GB after conversion, but I have no such disk capacity to allow more than 1500 users have 15GB quota. Would it be practical solution to compress all files after conversion and enable zlib support in dovecot? From amateo at um.es Wed Sep 12 17:18:47 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 12 Sep 2012 16:18:47 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <20120912113407.GG22070@vega.lgb.hu> References: <5049C861.2030904@um.es> <20120911222103.GB15502@vega.lgb.hu> <10D48D09-B3E9-4897-B8C8-0903C3483477@iki.fi> <20120912113407.GG22070@vega.lgb.hu> Message-ID: <505099C7.40709@um.es> El 12/09/12 13:34, G?bor L?n?rt escribi?: > On Wed, Sep 12, 2012 at 01:28:58AM +0300, Timo Sirainen wrote: >> Activating quota is equally slow, because it needs to scan sizes of all >> files. If the size is stored in maildir filename (,S=123) this is pretty >> fast, otherwise it's pretty slow because Dovecot stat()s the files, but >> afterwards it stores them to dovecot-uidlist file. So the main difference >> is that Maildir++ requires rescanning the quota periodically, while dict >> quota never rescans unless you run doveadm quota rescan. > > Ahaa, thanks for your answer. > > But ... why is rescan needed sometimes with maildir++? If I don't touch > maildir by other softwares ever (I even have maildir_very_dirty_syncs = yes) > just dovecot pop3/imap and incoming mails via dovecot's lmtp (no other MDAs, > not even the LDA, etc), then why is it needed to rescan periodically with > maildir? I don't really understand the difference then which needs quota > rescan with maildir++ but not with dict quota during the very same usage > environment otherwise. > > Anyway I guess, having dict quota with SQL backend is not so a bad idea: I > would be able to check incoming mails (at MXs) if target user will have > enough space to hold the mail by using the mail size and quota usage (from > SQL what dict quota uses as well) on the MXs using eg postfix policy server > ... So I won't generate ugly NDRs later, after accepted mails. Not 100% > accurate (quota usage may change while mail hits dovecot) but at least it > will stop the majority of those quota related NDRs I have problems with > currently. > > The only thing which keep me away from dict quota with sql backend that it > needs much more complex stuff than simply using maildir++ quota, but it's > maybe only my lazyness ... > You could also use dict quota stored in a file in the user's home directory as maildir++ quota does. This is the way I'm going to use it, not with a sql backend. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From lgb at lgb.hu Wed Sep 12 17:40:00 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:40:00 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb Message-ID: <20120912144000.GB1700@vega.lgb.hu> Hi, I'm about getting to know dovecot in details, I am about the command "doveadm user". The man page shows this example: userdb: jane uid : 8001 gid : 8001 home : /home/jane mail : sdbox:~/sdbox plugins : sieve quota_rule: *:storage=150M It seems to be nice, however I never got this only: ~# doveadm user lgb at office userdb: lgb at office That's all. The dovecot test configuration works otherwise via IMAP/POP3/LMTP nicely. What I guess that the lack of extra information (which would be needed by some scripts of mine, eg for getting user's home easily from shell scripts, as it's hash based, etc) is caused because I don't pass home (and/or other settings) back from userdb and/or passdb. However I do have these in global scope: mail_home = /mailstorage/%Ld/%Ln (yes, this is not hashed at all at the momement but it will be soon) mail_location = maildir:~/Maildir mail_uid = vmail mail_gid = vmail (... and also some global quota stuffs configured in plugin section) Now, I am a bit confused: even with passdb/userdb does not return with user home or GID/UID, dovecot knows these in case of pop3/imap/lmtp access, since these information can be "composed" from those settings. But then, wouldn't it possible so "doveadm user" shows those as well, like lmtp/pop3/imap can resolve those too? Honestly, it's a bit redundant to put needed information into user and pass attrs in every dbs (I have some) when dovecot knows those otherwise ... If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb query tool nothing more), is there any tool which works in this config, displaying extra information (at least user's home), or should I create some script which just repeats the functionality of dovecot's configured mail_home resolution? Thanks. From lgb at lgb.hu Wed Sep 12 17:48:52 2012 From: lgb at lgb.hu (=?iso-8859-1?B?R+Fib3IgTOlu4XJ0?=) Date: Wed, 12 Sep 2012 16:48:52 +0200 Subject: [Dovecot] question on "doveadm user" and pass/userdb In-Reply-To: <20120912144000.GB1700@vega.lgb.hu> References: <20120912144000.GB1700@vega.lgb.hu> Message-ID: <20120912144852.GA2372@vega.lgb.hu> On Wed, Sep 12, 2012 at 04:40:00PM +0200, G?bor L?n?rt wrote: > If I interpret "doveadm user"'s goal wrong (it's just for userdb/passdb > query tool nothing more), is there any tool which works in this config, > displaying extra information (at least user's home), or should I create some > script which just repeats the functionality of dovecot's configured > mail_home resolution? Arghhh, never mind, it seems switch -m does this :) However this is not documented in the man page, I just discovered because "doveadm user" (without user name) lists that, and I tried out what it does ... From kgc at corp.sonic.net Wed Sep 12 19:55:41 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 12 Sep 2012 09:55:41 -0700 Subject: [Dovecot] TIMO HELP! director ring wont stay connected In-Reply-To: <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> References: <5044F66F.3010505@corp.sonic.net> <38C2DCC3-B532-4DE4-AC60-1204FA412FE0@iki.fi> Message-ID: <5050BE8D.2090002@corp.sonic.net> Timo, I'm glad to hear that you found a plausible cause and have got a patch in to fix it. -K From silent_wumpus at yahoo.com Wed Sep 12 20:51:35 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 10:51:35 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> Message-ID: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From silent_wumpus at yahoo.com Wed Sep 12 22:14:24 2012 From: silent_wumpus at yahoo.com (Tim E.) Date: Wed, 12 Sep 2012 12:14:24 -0700 (PDT) Subject: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method In-Reply-To: <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> References: <1347046358.4985.YahooMailNeo@web121502.mail.ne1.yahoo.com> <2F4BA9F8-00C8-4C63-8B65-7C789C8F0BC2@iki.fi> <1347403140.89788.YahooMailNeo@web121506.mail.ne1.yahoo.com> <721D10D7-7B4D-46F8-8F01-269453F82A7D@iki.fi> <1347472295.97283.YahooMailNeo@web121503.mail.ne1.yahoo.com> Message-ID: <1347477264.9603.YahooMailNeo@web121506.mail.ne1.yahoo.com> Added these lines to /etc/dovecot.conf namespace { separator = / inbox = yes } Ran the command and it works. However it doesnt keep the UID as the thunderbird client redownloads all the POP3 mails. ________________________________ From: Tim E. To: Dovecot Mailing List Sent: Wednesday, September 12, 2012 1:51 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method > First of all, you mean you have only one mbox file per user?? The old server is a POP3 only. It does not support IMAP. On the old server each user has one Inbox in eg: /var/spool/mail/user <- 'user' being the the mbox file I moved the mbox file user into /home/trans, created /oldhome/user and ran your command as best I understood it. The results are below. [root at srv1 ~]# ls -lR /oldhome /oldhome: total 4 drwxrwxrwx 2 user root 4096 Sep 12 11:06 user /oldhome/user: total 0 [root at srv1 ~]# ls -lR /home/trans /home/trans: total 29992 -rwxrwxrwx 1 root root 30709562 Sep 12 09:53 user [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/user dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) [root at srv1 ~]# [root at srv1 ~]# dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { ? driver = pam } plugin { ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } protocols = imap pop3 ssl_cert = To: Tim E. Cc: Dovecot Mailing List Sent: Tuesday, September 11, 2012 6:48 PM Subject: Re: [Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method On 12.9.2012, at 1.39, Tim E. wrote: > Sorry for the delay. I have tried and failed to use dsync properly. What would the format and usage of the command be? > I have all my old Mbox mail file called user onto the new server under /home/trans and used dsync as follows: > > dsync -v -u user backup user at domain.com First of all, you mean you have only one mbox file per user? If the users used IMAP protocol, you should have more than one mbox. The idea is that you first configure Dovecot to point to maildirs, and then use for example: dsync -u user mirror mbox:/oldhome/user:INBOX=/home/trans/%u Where /oldhome/user/ would have the non-INBOX files. > All I get back is the usage. Also, another concern. Will the dovecot indexes be recreated as they where on the old server. You don't have to worry about indexes. > Any help would be appreciated. The real question is will the clients have to redownload their emails as I described in the first post. dsync should preserve all the UIDs and POP3 UIDLs, assuming you have the same pop3_uidl_format setting. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:38:45 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:38:45 +0200 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <50501E34.2030900@optusnet.com.au> References: <50501E34.2030900@optusnet.com.au> Message-ID: <20120913003845.GA10550@daniel.localdomain> Dominic Malolepszy wrote: > Is there a way to strip a backslash if it exists at the start of > username like \user at example.com within the Dovecot LDA? I am using > Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL > trickery. Address rewriting should be done in the MTA. I see no point why this should be done in the LDA? Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:47:04 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:47:04 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names Message-ID: <20120913004704.GA10738@daniel.localdomain> Hi, the Drafts special folder is called "Entw?rfe" in German locale. How should the special_use entry in the config file look like? Would you recommended to use the encoded folder name... mailbox "Entw&APw-rfe" { special_use = \Drafts } ...or rather the utf-8 Encoding in the dovecot config file? mailbox "Entw?rfe" { special_use = \Drafts } Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 03:54:57 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 02:54:57 +0200 Subject: [Dovecot] doveadm director map : User listing returned failure In-Reply-To: References: Message-ID: <20120913005457.GA10928@daniel.localdomain> Ram?n Frontera wrote: > we are running Dovecot 2.1.9. > We have 2 proxies with director and 4 mail servers. > We have the user information in a LDAP. > > I try to do: > # doveadm director map > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > user mail server ip expire time > x.x.x.x 2012-09-12 10:52:17 > x.x.x.x 2012-09-12 10:55:53 > > in the dovecot log i have found: > dovecot: auth: Error: Trying to iterate users, but userdbs don't support it Please see http://wiki2.dovecot.org/Tools/Doveadm/Mailbox Make sure that the iterate_attrs and iterate_filter settings in /etc/dovecot/dovecot-ldap.conf.ext match your LDAP schema and the user is allowed to iterate the LDAP tree. Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Thu Sep 13 09:34:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 13 Sep 2012 08:34:36 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <50517E7C.5070200@schetterer.org> Am 13.09.2012 02:47, schrieb Daniel Parthey: > Hi, > > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } > > Regards > Daniel > which german client does mailbox "Entw&APw-rfe" and is able to use special_use i ve testet k9 mail 4.2 german, no need for translate \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i think this or equal should be the right way for all clients i ve done this for Archive ( to goal Tb Layout, which hasnt special_use yet), but still untested, so try test something for like this for Drafts too, but for now i dont know a client wich is ready and out for special use with german language ( the new outlook will be, but its not out in german yet , as my latest look on it, the beta is only in englisch or spanish ) mailbox Archive { special_use = \Archive auto=no } mailbox "Archives" { special_use = \Archive auto=subscribe } login telnet . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE SEARCH=FUZZY COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in . LIST (SPECIAL-USE) "" "*" * LIST (\Trash) "/" "Trash" * LIST (\Junk) "/" "Junk" * LIST (\Sent) "/" "Sent" * LIST (\Drafts) "/" "Drafts" * LIST (\Archive) "/" "Archives" * LIST (\Unmarked) "/" "INBOX" -- Best Regards MfG Robert Schetterer From zinovik.igor at gmail.com Thu Sep 13 15:55:00 2012 From: zinovik.igor at gmail.com (Igor Zinovik) Date: Thu, 13 Sep 2012 16:55:00 +0400 Subject: [Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes In-Reply-To: References: Message-ID: Completly PEBKAC problem. I was not cleaning Maildir directory after several conversions. 2012/9/12 Igor Zinovik : > Hello, list. > > I'm asking question not directly connected with dovecot. > > I'm planning a migration from cyrus-imapd to dovecot and i'm using > cyrus2dovecot script (by Freie Universit?t Berlin) to convert mailboxes > of users, but i'm wondering why size of mailbox in Maildir++ is so > much bigger than mailbox in cyrus format after conversion: > linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik > /srv/vmail/petrsu.ru/z/zinovik/Maildir > 238M /mnt/imap/z/user/zinovik > 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir > > I was planning to implement quota for mailboxes about 1 gigabyte, > but after conversion I would not be able to receive messages to my own > box, because i'm overquota. > > I think the only way would be to set quota up to 15 GB since some of users > have mailbox with 12GB after conversion, but I have no such disk capacity > to allow more than 1500 users have 15GB quota. > > Would it be practical solution to compress all files after conversion and enable > zlib support in dovecot? From ben at indietorrent.org Thu Sep 13 20:25:18 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 13 Sep 2012 13:25:18 -0400 Subject: [Dovecot] antispam_plugin prevents IMAP login (error 3) [Dovecot 2.0.19] In-Reply-To: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> References: <22a789efa29d55b6725759b7ee4b5bd0@webmail.projektfabrik.com> Message-ID: <505216FE.20807@indietorrent.org> > On 17.08.2012 12:06, dof at projektfabrik.com wrote: >> Hi everybody, >> >> trying to get the Dovecot antispam_plugin to work and I must be doing >> something wrong, because as soon as it is enabled with a certain >> backend, imap logins do not work anymore (the session is immediately >> closed after a successful login). Interestingly, pipe and spool2dir >> are working (that is, the session won't be closed), dspam-exec and >> crm114-exec are not. If this happens, mail_debug output is as >> follows: >> >> [...] > > Hi again, > > how embarrassing. I should use the correct name for the backend type. > :\ So everything works as expected, sorry for spamming the mailing list. > > > Best regards, > Florian Florian, I am having a very similar (or the same) issue. Ubuntu 12.04.1 LTS with Dovecot 2.0.19. Do you mind sharing the details of your configuration, and what, exactly, you changed with respect to the back-end to resolve the issue? a.) Which back-end are you using? b.) What are your "plugin {}" configuration directives? I'm attempting to use mailtrain, per http://wiki2.dovecot.org/Plugins/Antispam . My configuration directives are: plugin { antispam_backend = mailtrain antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_spam_pattern_ignorecase = spam;junk } Thanks for any help! -Ben From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 13 22:23:08 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 13 Sep 2012 21:23:08 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <50517E7C.5070200@schetterer.org> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> Message-ID: <20120913192308.GA9400@daniel.localdomain> Robert Schetterer wrote: > > the Drafts special folder is called "Entw?rfe" in German locale. > > How should the special_use entry in the config file look like? > > > > Would you recommended to use the encoded folder name... > > > > mailbox "Entw&APw-rfe" { > > special_use = \Drafts > > } > > > > ...or rather the utf-8 Encoding in the dovecot config file? > > > > mailbox "Entw?rfe" { > > special_use = \Drafts > > } > > which german client does mailbox "Entw&APw-rfe" and is able to use special_use? > > i ve tested k9 mail 4.2 german, no need for translate > \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i > think this or equal should be the right way > for all clients I'm not sure which client created it and if any of my clients does support special_use, but I found the following mailbox on our mdbox filesystem: # ls -ld mail/mailboxes/Entw\&APw-rfe drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe Seems like k9mail or Android Mail must have created it in May 2012, the folder is not subscribed and contains only a single mail. The rest is in "Drafts" mailbox (where it belongs). Regards Daniel -- https://plus.google.com/103021802792276734820 From robert at schetterer.org Fri Sep 14 08:33:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 14 Sep 2012 07:33:25 +0200 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913192308.GA9400@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> <50517E7C.5070200@schetterer.org> <20120913192308.GA9400@daniel.localdomain> Message-ID: <5052C1A5.40409@schetterer.org> Am 13.09.2012 21:23, schrieb Daniel Parthey: > Robert Schetterer wrote: >>> the Drafts special folder is called "Entw?rfe" in German locale. >>> How should the special_use entry in the config file look like? >>> >>> Would you recommended to use the encoded folder name... >>> >>> mailbox "Entw&APw-rfe" { >>> special_use = \Drafts >>> } >>> >>> ...or rather the utf-8 Encoding in the dovecot config file? >>> >>> mailbox "Entw?rfe" { >>> special_use = \Drafts >>> } >> >> which german client does mailbox "Entw&APw-rfe" and is able to use special_use? >> >> i ve tested k9 mail 4.2 german, no need for translate >> \Drafts is auto noticed then it does \Drafts ( Entw?rfe ) in the gui, i >> think this or equal should be the right way >> for all clients > > I'm not sure which client created it and if any of my clients > does support special_use, but I found the following mailbox on > our mdbox filesystem: > > # ls -ld mail/mailboxes/Entw\&APw-rfe > drwx------ 3 vmail vmail 4096 2012-05-22 19:58 mail/mailboxes/Entw&APw-rfe > > Seems like k9mail or Android Mail must have created it in May 2012, > the folder is not subscribed and contains only a single mail. The > rest is in "Drafts" mailbox (where it belongs). > > Regards > Daniel > I would recommand stay cool until you can verify german clients using imap special_use i dont think it makes sense react now, Timo said he might will create some wiki page in relate clients matrix with special use -- Best Regards MfG Robert Schetterer From gcharot at evenium.com Fri Sep 14 11:59:57 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 10:59:57 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail Message-ID: <5052F20D.7050706@evenium.com> Hello all, I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot of lock and "sieve_execute_bytecode error" in my dovecot.log. $ cat /etc/redhat-release CentOS release 5.8 (Final) $ dovecot --version 1.0.13 The sieve scripts are generated by squirrelmail mail and are mostly mail filters and vacations notices. $ yum info squirrelmail.noarch Installed Packages Name : squirrelmail Arch : noarch Version : 1.4.8 Release : 5.el5.centos.13 When I look at my dovecot.log I have a lot of sieve error : deliver(foo): Sep 11 15:08:04 Info: msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: save failed to EvSUPPORT: Timeout while wait ing for lock deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: Generic Error deliver(foo): Sep 11 15:08:04 Error: sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed The server hosts around 50 mailboxes, some people does have sieve errors some doesn't. Is there any way to get more details about the runtime error ? How can I debug this issue ? Thanks a lot in advance, Greg My config below : #dovecot -n # 1.0.13: /etc/dovecot.conf log_path: /var/log/dovecot.log ssl_ca_file: /etc/pki/roch_crt/roch_evenium_com.ca-bundle ssl_cert_file: /etc/pki/roch_crt/roch_evenium_com.crt ssl_key_file: /etc/pki/roch_crt/private/roch-private.key ssl_cipher_list: ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+SSLv3 login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_process_per_connection: no login_process_size: 256 login_processes_count: 10 login_max_connections: 32 mail_privileged_group: mail dotlock_use_excl: yes fsync_disable: yes maildir_copy_with_hardlinks: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls,oe-ns-eoh auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 From amateo at um.es Fri Sep 14 12:13:03 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 14 Sep 2012 11:13:03 +0200 Subject: [Dovecot] best way to activate quota In-Reply-To: <5049CC5D.7070005@schetterer.org> References: <5049C861.2030904@um.es> <5049CC5D.7070005@schetterer.org> Message-ID: <5052F51F.3040605@um.es> El 07/09/12 12:28, Robert Schetterer escribi?: >> >> I have tried to activate it in one of my nodes. The problem is that >> load of it has increase a lot, as much as the system was unusable >> (maildir is in nfs storage, with indexes in local disks). > > that might not be the optimal storage solution > Why do you say it might be not optimal? I think that nfs storage is a very common scenario. From carsten.delellis at delellis.net Fri Sep 14 12:27:23 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Fri, 14 Sep 2012 11:27:23 +0200 Subject: [Dovecot] Couldn't drop privileges: User is missing UID (see mail_uid setting) Message-ID: Hi all I hope anyone could help me out with my problem regarding the user_id settings. I am running my mail servers just for fun and I am not a professional mail admin. After I am using postfix and dovecot for my mail servers with a mysql backend for the last 2 years I have tried to set up an installation with a ldap backend. I followed the how-to from the ?Postfix Book? and now I have always the following error message: Couldn't drop privileges: User is missing UID (see mail_uid setting) This error message occurs whenever I try to connect via a mail client. System emails were delivered the way I expected. So I assume postfix is working fine. The ldap search I use for user authentication is as follows: uris = ldap://myserver.com dn = cn=admin, dc=mydomain, dc=com dnpass = secure tls = no base = ou=mailuser,dc=mydomain,dc=com ldap_version = 3 user_attrs = pfMailMessageStore=home user_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) pass_attrs = pfClearPassword=password,pfMailMessageStore=userdb_home pass_filter = (&(objectClass=PFMailAccount)(pfMail=%u)) default_pass_scheme = PLAIN user_global_uid = 5000 user_global_gid = 5000 I am using dovecot 2.1.9, postfix 2.8.5 and openldap 2.4.23 I would appreciate any help or tipps. ~ Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From stephan at rename-it.nl Fri Sep 14 12:48:21 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 11:48:21 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052F20D.7050706@evenium.com> References: <5052F20D.7050706@evenium.com> Message-ID: <5052FD65.8020104@rename-it.nl> Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: > Hello all, > > I'm using dovecot on CentOS 5.8 (official CentOS package) and have lot > of lock and "sieve_execute_bytecode error" in my dovecot.log. > > $ cat /etc/redhat-release > CentOS release 5.8 (Final) > > $ dovecot --version > 1.0.13 Very, very, very, very old. > The sieve scripts are generated by squirrelmail mail and are mostly > mail filters and vacations notices. > > $ yum info squirrelmail.noarch > Installed Packages > Name : squirrelmail > Arch : noarch > Version : 1.4.8 > Release : 5.el5.centos.13 > > When I look at my dovecot.log I have a lot of sieve error : > > deliver(foo): Sep 11 15:08:04 Info: > msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: > save failed to EvSUPPORT: Timeout while wait > ing for lock There's the actual error. This has very little to do with Sieve itself. Looks like some mbox locking issue, but I don't really know. You could try enabling mail_debug to get more info. > deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: > Generic Error > deliver(foo): Sep 11 15:08:04 Error: > sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed Yuck, I was hoping that people would finally have stopped using cmusieve. Regards, Stephan. From Jost.Krieger+dovecot at rub.de Fri Sep 14 13:47:59 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 14 Sep 2012 12:47:59 +0200 Subject: [Dovecot] Possible mbox corruption In-Reply-To: References: <20120831084233.GE3002@ruhr-uni-bochum.de> <20120903141121.GR3002@ruhr-uni-bochum.de> Message-ID: <20120914104758.GU30629@ruhr-uni-bochum.de> On Tue 11 Sep 2012 05:33:10 PM GMT, Timo Sirainen wrote: > Any Content-Length header sent by IMAP client or via LDA input should > always get dropped (among some other headers) and replaced with the > correct value. So if there's a bug, I doubt it has anything to do with > that. But if you can easily reproduce this failure it would be nice to > get it fixed.. You're right, of course. And I cannot reproduce the problem. The user that could reproduce it has somehow given up. I'v experimented and read the source, and there is no way a "Content-Length" could be saved by dovecot :-) But ... This is on a system where delivery is not (yet) via dovecot, but directly by qmail. Two possibilities come to mind: 1. We have a locking problem that only surfaces when qmail and dovecot try to write at the same time to the end of the mbox. Most people (and clients) hardly ever do an APPEND to the INBOX. 2. Something qmail writes to the INBOX makes dovecot hiccup (like a wrong Content-Length header :-;). Which would anyone think more likely? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From gcharot at evenium.com Fri Sep 14 14:03:25 2012 From: gcharot at evenium.com (Gregory Charot (EVENIUM)) Date: Fri, 14 Sep 2012 13:03:25 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <5052FD65.8020104@rename-it.nl> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> Message-ID: <50530EFD.8080907@evenium.com> Thanks for the quick reply Stephan ! Is it possible to upgrade from 1.0.13 (which is the official CentOS release) to the atrpms version 2.1.1-2 available at http://packages.atrpms.net/dist/el5/dovecot/ Do I need a migration plan ? Thanks ! Greg Le 14/09/2012 11:48, Stephan Bosch a ?crit : > Op 9/14/2012 10:59 AM, Gregory Charot (EVENIUM) schreef: >> Hello all, >> >> I'm using dovecot on CentOS 5.8 (official CentOS package) and have >> lot of lock and "sieve_execute_bytecode error" in my dovecot.log. >> >> $ cat /etc/redhat-release >> CentOS release 5.8 (Final) >> >> $ dovecot --version >> 1.0.13 > > Very, very, very, very old. > >> The sieve scripts are generated by squirrelmail mail and are mostly >> mail filters and vacations notices. >> >> $ yum info squirrelmail.noarch >> Installed Packages >> Name : squirrelmail >> Arch : noarch >> Version : 1.4.8 >> Release : 5.el5.centos.13 >> >> When I look at my dovecot.log I have a lot of sieve error : >> >> deliver(foo): Sep 11 15:08:04 Info: >> msgid=<1881992580.96697.1347363282479.JavaMail.jboss at gula1.xxx.com>: >> save failed to EvSUPPORT: Timeout while wait >> ing for lock > > There's the actual error. This has very little to do with Sieve > itself. Looks like some mbox locking issue, but I don't really know. > You could try enabling mail_debug to get more info. > >> deliver(foo): Sep 11 15:08:04 Info: sieve runtime error: Fileinto: >> Generic Error >> deliver(foo): Sep 11 15:08:04 Error: >> sieve_execute_bytecode(/home/foo/.dovecot.sievec) failed > > Yuck, I was hoping that people would finally have stopped using cmusieve. > > Regards, > > Stephan. From stephan at rename-it.nl Fri Sep 14 14:37:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 14 Sep 2012 13:37:07 +0200 Subject: [Dovecot] sieve_execute_bytecode failed with squirrelmail In-Reply-To: <50530EFD.8080907@evenium.com> References: <5052F20D.7050706@evenium.com> <5052FD65.8020104@rename-it.nl> <50530EFD.8080907@evenium.com> Message-ID: <505316E3.5010802@rename-it.nl> Op 9/14/2012 1:03 PM, Gregory Charot (EVENIUM) schreef: > Thanks for the quick reply Stephan ! > > Is it possible to upgrade from 1.0.13 (which is the official CentOS > release) to the atrpms version 2.1.1-2 available at > > http://packages.atrpms.net/dist/el5/dovecot/ > > Do I need a migration plan ? http://wiki2.dovecot.org/Upgrading?action=show&redirect=UpgradingDovecot http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Migration Regards, Stephan. From roundcube222 at alaadin.org Fri Sep 14 13:37:31 2012 From: roundcube222 at alaadin.org (Robert JR) Date: Fri, 14 Sep 2012 13:37:31 +0300 Subject: [Dovecot] POP3S PORT Message-ID: Hello I am using dovecot with POP3S and IMAPS ports, When client connect to POP3S port (995) , i can see in dovecot log file that the protocol is TLSv1 So how come the protocol is TLSv1 and not SSL ? Does the user connect t port 995 with SSL protocol then encrypt again inside the SSL session to make The TLSv1 protocol ? Or does the client use TLSv1 Protocol immediately when connect to port 995 without Using SSL protocol ? Please Help. Robert JR From mailinglist at darac.org.uk Fri Sep 14 14:49:15 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 14 Sep 2012 12:49:15 +0100 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <20120914114915.GA23431@darac.org.uk> On Fri, Sep 14, 2012 at 01:37:31PM +0300, Robert JR wrote: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? First of all, read http://wiki2.dovecot.org/SSL and see if that answers your question. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 14 15:06:32 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 14 Sep 2012 14:06:32 +0200 Subject: [Dovecot] POP3S PORT In-Reply-To: References: Message-ID: <50531DC8.207@thelounge.net> Am 14.09.2012 12:37, schrieb Robert JR: > > > Hello > > I am using dovecot with POP3S and IMAPS ports, > > When client > connect to POP3S port (995) , i can see in dovecot log file that the > protocol is TLSv1 > > So how come the protocol is TLSv1 and not SSL ? because it is simply the same read about what TLS is -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From damoxc at gmail.com Fri Sep 14 17:46:39 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 15:46:39 +0100 Subject: [Dovecot] key -> object mailstore Message-ID: Hi, I was wondering what would be entailed in modifying dovecot 2.2 to support storing mail in an object store. I've seen a few mails dotted around in the ML history about supporting such a thing and seen it's basically dependant upon some changes in lib-storage to support writing messages without locking. Is this still the case? Regards, Damien From tss at iki.fi Fri Sep 14 17:59:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 14 Sep 2012 17:59:17 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: Message-ID: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> On 14.9.2012, at 17.46, Damien Churchill wrote: > I was wondering what would be entailed in modifying dovecot 2.2 to > support storing mail in an object store. I've seen a few mails dotted > around in the ML history about supporting such a thing and seen it's > basically dependant upon some changes in lib-storage to support > writing messages without locking. Is this still the case? I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? The generic idea is: - only one server accesses one user simultaneously - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync - support high latency: asynchronous reads/writes. prefetch mail bodies. From damoxc at gmail.com Fri Sep 14 18:16:09 2012 From: damoxc at gmail.com (Damien Churchill) Date: Fri, 14 Sep 2012 16:16:09 +0100 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: On 14 September 2012 15:59, Timo Sirainen wrote: > On 14.9.2012, at 17.46, Damien Churchill wrote: > >> I was wondering what would be entailed in modifying dovecot 2.2 to >> support storing mail in an object store. I've seen a few mails dotted >> around in the ML history about supporting such a thing and seen it's >> basically dependant upon some changes in lib-storage to support >> writing messages without locking. Is this still the case? > > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? > That's good to hear :) I've been evaluating RADOS as an object store, which is similar to S3. Although any distributed storage would be great. I'd be more than happy to help code it! > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. > I'm assuming that the director would be used in order to distribute connections to the same server, so it's only within a local instance of dovecot you'd need to be aware of what currently has a connection open for that user? How are you planning on handling the situation where say node X dies and hasn't uploaded the latest index file? Would that result in missing messages from the mailbox when accessed by another node, or is the local index intended to be more of a write-through cache? From ncjeffgus at zimage.com Sat Sep 15 06:39:36 2012 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Fri, 14 Sep 2012 20:39:36 -0700 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <1347680376.5717.61.camel@maclinux> On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it > for v2.2. Do you want to help coding it? :) Which storage would you > want to use? > > The generic idea is: > - only one server accesses one user simultaneously > - index files are copied from object storage to local filesystem and > accessed there, once in a while uploaded back to object storage > - if user is accessed from two servers because of some bug/split > brain/something, the changes are merged using dsync > - support high latency: asynchronous reads/writes. prefetch mail bodies. With this system, would the read/write ultimately go to a normal OS file function? If it is a file function, could this be used with a system like glusterfs, ceph, etc? The other option would be to write it against a object store client library and bypass the normal file functions. ...Jeff From patrickdk at patrickdk.com Sat Sep 15 19:50:47 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sat, 15 Sep 2012 12:50:47 -0400 Subject: [Dovecot] v2.1 memory usage In-Reply-To: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> References: <4AC709BB-5749-49AD-8C93-69AD2A70D25A@iki.fi> Message-ID: <20120915125047.Horde.iNAEcZLnE6FQVLHnJ7HDDyA@mail.patrickdk.com> Only have 650 imap clients this weekend, have to give it a try during a weekday. Total(MB) Avg(kB) Name 2 4 /lib/ld-2.11.1.so 2 4 /lib/libbz2.so.1.0.4 2 4 /lib/libc-2.11.1.so 63 96 /lib/libcrypto.so.0.9.8 2 4 /lib/libdl-2.11.1.so 0 0 /lib/libnss_dns-2.11.1.so 0 0 /lib/libnss_files-2.11.1.so 2 4 /lib/libpthread-2.11.1.so 0 0 /lib/libresolv-2.11.1.so 2 4 /lib/librt-2.11.1.so 13 20 /lib/libssl.so.0.9.8 2 4 /lib/libz.so.1.2.3.3 2 4 /usr/lib/dovecot/imap 10 16 /usr/lib/dovecot/libdovecot-storage.so.0.0.0 2 4 /usr/lib/dovecot/libdovecot.so.0.0.0 2 4 /usr/lib/dovecot/modules/lib10_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib20_zlib_plugin.so 2 4 /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so 435 661 [anonymous] 640 973 [heap] 55 84 [stack] 1253 58352 TOTAL Quoting Timo Sirainen : > v2.1's imap/pop3 processes are linked with OpenSSL by default, > because of imapc/pop3c storages. I wonder if you notice any actual > (system-global) memory usage reduction when compiling without those > and you have thousands of imap processes? > > ./configure --with-storages=mdbox sdbox maildir mbox cydir > > I guess I could try it myself also but maybe someone else wanting to > reduce memory usage is quicker? :) > > Related to that, you can look at where imap processes' memory goes > with: http://dovecot.org/tools/linux-maps-parse-imap.pl > I don't know how much memory those (writable!) file-backed memory > areas actually use per-process. Maybe nothing, maybe something.. From carsten.delellis at delellis.net Sun Sep 16 13:48:17 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 12:48:17 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database Message-ID: Hi all I receive the following error in my mail.log. Does anybody know how to configure dovecot to get rid of this ? I have one server up and running with postfix, dovecot and a mysql backend. The error above happens on another server with postfix, dovecot and a ldap backend. Any hints ? Mit freundlichem Gru? Carsten Laun-De Lellis Dipl.-Ing. Elektrotechnik Certified Information Systems Auditor (CISA) Hauptstrasse 13 D-67705 Trippstadt Phone: +49 (6306) 992140 Mobile: +49 (151) 27530865 Fax: +49 (6306) 992142 email: carsten.delellis at delellis.net From user+dovecot at localhost.localdomain.org Sun Sep 16 14:21:11 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Sun, 16 Sep 2012 13:21:11 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <5055B627.9010809@localhost.localdomain.org> On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > Hi all > > I receive the following error in my mail.log. > > Does anybody know how to configure dovecot to get rid of this ? > [?] Fix your setup. ,--[ http://wiki2.dovecot.org/UserDatabase ]-- | The user database lookup returns these fields: | ? | * home: User's home directory, overrides the global mail_home setting. | ? `-- See also: http://wiki2.dovecot.org/VirtualUsers/Home Regards, Pascal -- The trapper recommends today: cafefeed.1226013 at localdomain.org From noel.butler at ausics.net Sun Sep 16 14:32:01 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:32:01 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <5055B627.9010809@localhost.localdomain.org> References: <5055B627.9010809@localhost.localdomain.org> Message-ID: <1347795121.3768.5.camel@tardis> thats a rather intelligent response now, isnt it troll. On Sun, 2012-09-16 at 13:21 +0200, Pascal Volk wrote: > On 09/16/2012 12:48 PM Carsten Laun-De Lellis wrote: > > Hi all > > > > I receive the following error in my mail.log. > > > > Does anybody know how to configure dovecot to get rid of this ? > > [?] > > Fix your setup. > > ,--[ http://wiki2.dovecot.org/UserDatabase ]-- > | The user database lookup returns these fields: > | ? > | * home: User's home directory, overrides the global mail_home setting. > | ? > `-- > > See also: http://wiki2.dovecot.org/VirtualUsers/Home > > > Regards, > Pascal -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Sun Sep 16 14:33:33 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sun, 16 Sep 2012 21:33:33 +1000 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: References: Message-ID: <1347795213.3768.8.camel@tardis> Hi, On Sun, 2012-09-16 at 12:48 +0200, Carsten Laun-De Lellis wrote: > Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a ldap > backend. should use mysql :) -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From mostafa.rokooie at gmail.com Sun Sep 16 16:44:49 2012 From: mostafa.rokooie at gmail.com (Mostafa Rokooie) Date: Sun, 16 Sep 2012 18:14:49 +0430 Subject: [Dovecot] Edit an existing message or replacing it Message-ID: Hi, I want to know is there any solution to edit an existing message in mailbox via IMAP? or replace it with a new message (store a new message with same UID)? Thank you very much, --Mostafa Rokooie From carsten.delellis at delellis.net Sun Sep 16 17:30:36 2012 From: carsten.delellis at delellis.net (Carsten Laun-De Lellis) Date: Sun, 16 Sep 2012 16:30:36 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database In-Reply-To: <1347795213.3768.8.camel@tardis> References: <1347795213.3768.8.camel@tardis> Message-ID: Hi First of all thank you for helping me to fix the problem. I am trying out ldap to get used to it. As I wrote I have a mysql setup up and running. Here's the output of doveconf -n: # 2.1.9: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-028stab099.3 x86_64 Ubuntu 11.04 vzfs auth_mechanisms = plain login disable_plaintext_auth = no hostname = smtp.example.com mail_location = maildir:/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/vmail/%d/%n prefix = separator = / subscriptions = yes type = private } passdb { args = /scripts/ldap/mail/auth.ldap driver = ldap } plugin { recipient_delimiter = + sieve = /vmail/%d/%n/dovecot.sieve sieve_dir = /vmail/%d/%n sieve_global_dir = /var/lib/dovecot/sieve sieve_global_path = /var/lib/dovecot/sieve/default.sieve } postmaster_address = root at example.com protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0660 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl_cert = Hi all > > > > I receive the following error in my mail.log. > Dont use ldap, but, you should supply the output of dovecot -n it might help those familiar with ldap setups assist you > > > Does anybody know how to configure dovecot to get rid of this ? > > > > I have one server up and running with postfix, dovecot and a mysql backend. > The error above happens on another server with postfix, dovecot and a > ldap backend. should use mysql :) From CMarcus at Media-Brokers.com Sun Sep 16 17:41:10 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 16 Sep 2012 10:41:10 -0400 Subject: [Dovecot] clucene build warnings? Message-ID: <5055E506.7040704@Media-Brokers.com> Hi all/Timo, Is clucene still the best/recommended method for enabling full text search? Although I haven't enabled FTS yet, I plan on doing sometime in the future, and was going to be using clucene, but before I do, I'd like to see if I need to worry about an issue I've been seeing ever since I first installed clucene (and still get every time it is upgraded)... I run gentoo linux on an amd64 server, and have always gotten this warning whenever it is updated (and also got it when I first installed it): QA Notice: Package triggers severe warnings which indicate that it may exhibit random runtime failures. /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? Please do not file a Gentoo bug and instead report the above QA issues directly to the upstream developers of this software. Homepage:http://clucene.sourceforge.net/ Has anyone else ever seen similar warnings when building clucene? Is the gentoo ebuild output correct that this is not a gentoo bug, but is in fact a problem with upstream? If so, is it harmless? Or should I try to get upstream to fix it? That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? Thanks, -- Best regards, Charles From sven at svenhartge.de Sun Sep 16 18:20:48 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 16 Sep 2012 17:20:48 +0200 Subject: [Dovecot] Error: User doesn't have home dir set, disabling duplicate database References: <1347795213.3768.8.camel@tardis> Message-ID: <6947fe6hfsv8@mids.svenhartge.de> Carsten Laun-De Lellis wrote: > user_attrs = home=/vmail/%d/%n,=uid=5000,=gid=5000 This should be user_attrs = =home=/vmail/%d/%n,=uid=5000,=gid=5000 You need the = before "home" like you did with "uid" and "gid" because you are not assigning any LDAP attribute to home but are construction one yourself with %d and %n. Or you can use the default_fields stanza in your userdb configuration to provide those values. user_attrs would then be empty. Gr??e, Sven. -- Sigmentation fault. Core dumped. From lists at kokelnet.de Sun Sep 16 18:48:28 2012 From: lists at kokelnet.de (Tobias Hachmer) Date: Sun, 16 Sep 2012 17:48:28 +0200 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. Message-ID: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Hello list, I am using dovecot v2.1.9 and have a problem with doveadm backup: # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here my namespace config: namespace { list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEX=/var/vmail/%d/%n/shared/%%u prefix = Gemeinsame-Postfaecher/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u prefix = separator = / subscriptions = yes type = private } Have I got a problem with my namespace config? Regards, Tobias Hachmer From slusarz at curecanti.org Sun Sep 16 21:55:39 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Sun, 16 Sep 2012 12:55:39 -0600 Subject: [Dovecot] Edit an existing message or replacing it In-Reply-To: References: Message-ID: <20120916125539.Horde.bh8AKIF5lbhQViCrQKfX2pA@bigworm.curecanti.org> Quoting Mostafa Rokooie : > I want to know is there any solution to edit an existing message in mailbox > via IMAP? or replace it with a new message (store a new message with same > UID)? You **can't** alter an existing UID message. You have to re-add the message to the mailbox and use a new UID. For this, you have 2 options: 1.) FETCH the message, alter it locally, APPEND the message to the server, and DELETE the original message. 2.) Use CATENATE to create a new message, and DELETE the original message (this is how we strip attachments from a message). Note that not all IMAP4rev1 servers support CATENATE. michael From dmalolepszy at optusnet.com.au Mon Sep 17 03:49:25 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 17 Sep 2012 10:49:25 +1000 Subject: [Dovecot] Strip backslash from usernames In-Reply-To: <20120913003845.GA10550@daniel.localdomain> References: <50501E34.2030900@optusnet.com.au> <20120913003845.GA10550@daniel.localdomain> Message-ID: <50567395.1040001@optusnet.com.au> On 13/09/12 10:38, Daniel Parthey wrote: > Dominic Malolepszy wrote: >> Is there a way to strip a backslash if it exists at the start of >> username like \user at example.com within the Dovecot LDA? I am using >> Dovecot 2.1.9 with a LDAP backend, so I cannot do any sort of SQL >> trickery. > > Address rewriting should be done in the MTA. > I see no point why this should be done in the LDA? > > Regards > Daniel Agreed, I was just hoping I would not have to poke around the existing Sendmail setup we have here, in the end it turned out the backslashes were leftover legacy crud in some of the LDAP entries that actually did nothing. From crohmann at netcologne.de Mon Sep 17 09:52:37 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Mon, 17 Sep 2012 08:52:37 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <5056C8B5.3070706@netcologne.de> Hey dovecot-users, On 14.09.2012 16:59, Timo Sirainen wrote: > I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? I'd vote for OpenStack's Swift or Ceph's RADOS. They are both gaining momentum with new instrallations, they are open source and quite active in development. Also they both maintain Amazon S3 compatible APIs. Ceph even has a Swift compatible API for that matter. Regards Christian From alessio at skye.it Mon Sep 17 10:09:09 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 17 Sep 2012 09:09:09 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment Message-ID: <5056CC95.8030209@skye.it> Hi, I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. Dovecot's wiki suggest to enable acl like this: plugin { # assumes mailboxes are in /var/mail/%d/%n: acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db } but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. So, why don't provide a way to restrict shared mailboxes also for dict in SQL? One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. Are there other possible solutions? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From tom at whyscream.net Mon Sep 17 11:40:11 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 17 Sep 2012 10:40:11 +0200 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <5056E1EB.5000600@whyscream.net> On 9/16/12 4:41 PM, Charles Marcus wrote: > Hi all/Timo, > > Is clucene still the best/recommended method for enabling full text search? > > Although I haven't enabled FTS yet, I plan on doing sometime in the > future, and was going to be using clucene, but before I do, I'd like to > see if I need to worry about an issue I've been seeing ever since I > first installed clucene (and still get every time it is upgraded)... > > I run gentoo linux on an amd64 server, and have always gotten this > warning whenever it is updated (and also got it when I first installed it): > > QA Notice: Package triggers severe warnings which indicate that it > may exhibit random runtime failures. > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: > warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, > int, size_t)? > Please do not file a Gentoo bug and instead report the above QA > issues directly to the upstream developers of this software. > Homepage:http://clucene.sourceforge.net/ > > Has anyone else ever seen similar warnings when building clucene? > > Is the gentoo ebuild output correct that this is not a gentoo bug, but > is in fact a problem with upstream? > > If so, is it harmless? Or should I try to get upstream to fix it? Same warnings here (gentoo 32bit). The gentoo package management simply checks the gcc output for warnings, and reports them to you after the build completes. As long as the warnings are not related to patches added by gentoo, the issue should go upstream. Whether it is a serious issue, that can only be decided by reviewing the code itself, which would be more appropriate for a lucene dev than a dovecot dev ;) I suggest that you simply report the issue upstream and see what happens. If the code is harmless, they might just change it the so that gcc doesn't complain about it in the future. -- Tom From CMarcus at Media-Brokers.com Mon Sep 17 12:56:35 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 05:56:35 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5056E1EB.5000600@whyscream.net> References: <5055E506.7040704@Media-Brokers.com> <5056E1EB.5000600@whyscream.net> Message-ID: <5056F3D3.4060006@Media-Brokers.com> On 2012-09-17 4:40 AM, Tom Hendrikx wrote: > I suggest that you simply report the issue upstream and see what > happens. If the code is harmless, they might just change it the so that > gcc doesn't complain about it in the future. Thanks Tom, but I already did, a long time ago (when I first installed it): https://sourceforge.net/tracker/?func=detail&aid=3494798&group_id=80013&atid=558446 Didn't hear anything for a month, and when I posted a follow-up, was told they were busy, and would take a look in the next few days, but nothing since and that was back in March... I just posted another follow-up, so we'll see what happens. I was mainly interested to know if anyone else sees this warning when compiling on other systems... -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 13:02:51 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:02:51 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) Message-ID: Hello, I have a quite common dovecot configuration with virtual users support: 1. User configuration is stored in MySQL database. 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. Is it possible to change mailbox mode to 0660? Dovecot version is 2.1.7 Part of SQL configuration: user_query = \ SELECT \ CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home, \ 500 AS uid, \ 500 AS gid \ FROM users u \ LEFT JOIN domains d ON u.domain_id = d.id \ WHERE \ u.user_name = '%n' AND \ d.domain_name = '%d' AND \ u.active = 1 Regards. -- M. From marcin at rootnode.net Mon Sep 17 13:09:22 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 12:09:22 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <2c3a5a4b367a9062389dfeb36d2cba61@rootnode.net> On 2012-09-17 12:02, Marcin H?ybin wrote: > 1. User configuration is stored in MySQL database. Obviously, I meant 0700 to be changed to 0770. -- M. From tss at iki.fi Mon Sep 17 15:21:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:21:51 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <5055E506.7040704@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> Message-ID: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> On 16.9.2012, at 17.41, Charles Marcus wrote: > /var/tmp/portage/dev-cpp/clucene-2.3.3.4-r4/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:129:33: warning: passing NULL to non-pointer argument 2 of ?void* memset(void*, int, size_t)? That's wrong, but it works anyway as intended. > That said, being that ianap, I probably wouldn't be much help to them, so, Timo, any chance you can see if this is a problem that needs to be addressed with them? CLucene seems to be a pretty dead project. They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. From tss at iki.fi Mon Sep 17 15:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:36:32 +0300 Subject: [Dovecot] RFC 6154: German umlauts in special_use folder names In-Reply-To: <20120913004704.GA10738@daniel.localdomain> References: <20120913004704.GA10738@daniel.localdomain> Message-ID: <55461F54-DC83-476B-910C-D8783E675DE8@iki.fi> On 13.9.2012, at 3.47, Daniel Parthey wrote: > the Drafts special folder is called "Entw?rfe" in German locale. > How should the special_use entry in the config file look like? > > Would you recommended to use the encoded folder name... > > mailbox "Entw&APw-rfe" { > special_use = \Drafts > } > > ...or rather the utf-8 Encoding in the dovecot config file? > > mailbox "Entw?rfe" { > special_use = \Drafts > } UTF8 is (or should be) now used everywhere in config files. From tss at iki.fi Mon Sep 17 15:40:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:40:15 +0300 Subject: [Dovecot] doveadm backup - Fatal: All your namespaces have a location setting. In-Reply-To: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> References: <8000d6bab09b172e1ff2a2251d82ddef@hachmer.de> Message-ID: <50602815-DD35-4B6B-B319-AA9261AC0812@iki.fi> On 16.9.2012, at 18.48, Tobias Hachmer wrote: > I am using dovecot v2.1.9 and have a problem with doveadm backup: > > # doveadm backup -u user at domain.com maildir:/backup/domain.com/user/Maildir > doveadm(user at domain.com): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Different installations use different kinds of namespaces for different purposes, so dsync can't simply sync all the namespaces. It syncs only the namespace which uses the global mail_location setting (not overriding it by having an explicit location setting). > namespace inbox { > inbox = yes > location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/indexes/%u So you probably should remove this location setting (and make sure mail_location is the same). > prefix = > separator = / > subscriptions = yes > type = private > } From CMarcus at Media-Brokers.com Mon Sep 17 15:42:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 08:42:32 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> Message-ID: <50571AB8.3010505@Media-Brokers.com> On 2012-09-17 8:21 AM, Timo Sirainen wrote: > CLucene seems to be a pretty dead project. Bummer... Was my memory correct that lucene is what you currently recommend for FTS for dovecot? > They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. Yuck - looks like it is java based? http://lucene.apache.org/ -- Best regards, Charles From tss at iki.fi Mon Sep 17 15:44:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:44:21 +0300 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <5056CC95.8030209@skye.it> References: <5056CC95.8030209@skye.it> Message-ID: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> On 17.9.2012, at 10.09, Alessio Cecchi wrote: > I would like to enable mailbox sharing for my users. I'm using dovecot 2.1.9 with vpopmail authentication. For security reason shared mailboxes should be available only for users of the same domain. > > Dovecot's wiki suggest to enable acl like this: > > plugin { > # assumes mailboxes are in /var/mail/%d/%n: > acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db > } > > but in vpopmail, and also in many others configuration, mailboxes are, for example, in /var/mail/nas1/%d, /var/mail/nas2/%d and so on, so it's difficult to restrict shared mailboxes available only for the same domain. A trick could be set acl_shared_dict to /var/mail/%d-shared-mailboxes.db, but I don't know if it's safe on an NFS environment. As long as all the servers have access to the file it doesn't matter where it is. You could even do something like /var/mail/nas1/shared-dict/%d.db > So, why don't provide a way to restrict shared mailboxes also for dict in SQL? > > One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. From tss at iki.fi Mon Sep 17 15:45:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:45:28 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50571AB8.3010505@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> Message-ID: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> On 17.9.2012, at 15.42, Charles Marcus wrote: > On 2012-09-17 8:21 AM, Timo Sirainen wrote: >> CLucene seems to be a pretty dead project. > > Bummer... > > Was my memory correct that lucene is what you currently recommend for FTS for dovecot? Lucene or Solr (i.e. anything but Squat). >> They've talked something about merging with Lucene++, but that hasn't happened. I've been wondering about moving to Lucene++. > > Yuck - looks like it is java based? > > http://lucene.apache.org/ Both CLucene and Lucene++ are C++ ports of the original Java Lucene. From tss at iki.fi Mon Sep 17 15:46:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:46:38 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: Message-ID: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> On 17.9.2012, at 13.02, Marcin H?ybin wrote: > I have a quite common dovecot configuration with virtual users support: > 1. User configuration is stored in MySQL database. > 2. Mailboxes are owned by vmail:vmail with permission 0600 or drwx------. > > Is it possible to change mailbox mode to 0660? Yeah, although since they're already sharing UIDs I don't see much point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway describes how it all works. From tss at iki.fi Mon Sep 17 15:57:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 15:57:57 +0300 Subject: [Dovecot] key -> object mailstore In-Reply-To: References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> Message-ID: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> On 14.9.2012, at 18.16, Damien Churchill wrote: > On 14 September 2012 15:59, Timo Sirainen wrote: >> On 14.9.2012, at 17.46, Damien Churchill wrote: >> >>> I was wondering what would be entailed in modifying dovecot 2.2 to >>> support storing mail in an object store. I've seen a few mails dotted >>> around in the ML history about supporting such a thing and seen it's >>> basically dependant upon some changes in lib-storage to support >>> writing messages without locking. Is this still the case? >> >> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >> > > That's good to hear :) I've been evaluating RADOS as an object store, > which is similar to S3. Although any distributed storage would be > great. I'd be more than happy to help code it! I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > > I'm assuming that the director would be used in order to distribute > connections to the same server, so it's only within a local instance > of dovecot you'd need to be aware of what currently has a connection > open for that user? Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > How are you planning on handling the situation where say node X dies > and hasn't uploaded the latest index file? Would that result in > missing messages from the mailbox when accessed by another node, or is > the local index intended to be more of a write-through cache? No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) From marcin at rootnode.net Mon Sep 17 15:57:31 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 14:57:31 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <8b54646de4ef445102e8f40ee8115c83@rootnode.net> On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? I have a service running with different UID that must have full access to mail folders. UID/user is added to vmail group. That is why 0770 permission is desired. -- M. From marcin at rootnode.net Mon Sep 17 16:25:27 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:25:27 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: On 2012-09-17 14:46, Timo Sirainen wrote: > Yeah, although since they're already sharing UIDs I don't see much > point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway > describes how it all works. Already tested and it does not work. Directory structure is /home/mail/// drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support Two last directories for mailbox support at maildomain.com are created automatically after first e-mail comes to this address. As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). Maybe additional configuration is needed to make it work? -- M. From tss at iki.fi Mon Sep 17 16:30:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 16:30:59 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> Message-ID: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> On 17.9.2012, at 16.25, Marcin H?ybin wrote: > On 2012-09-17 14:46, Timo Sirainen wrote: >> Yeah, although since they're already sharing UIDs I don't see much >> point? http://wiki2.dovecot.org/SharedMailboxes/Permissions anyway >> describes how it all works. > > Already tested and it does not work. > Directory structure is /home/mail/// > > drwxr-xr-x 8 root root 4096 Sep 14 10:28 /home > drwx--x--x 565 vmail vmail 53248 Sep 17 10:46 /home/mail > drwxrwx--- 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234 > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com > drwx------ 3 vmail vmail 4096 Sep 17 15:17 /home/mail/1234/maildomain.com/support > > Two last directories for mailbox support at maildomain.com are created automatically > after first e-mail comes to this address. > > As you can see root directory has 0770 permissions (or 0711 if /home/mail is the root directory). > Maybe additional configuration is needed to make it work? What's your mail_location setting? Note the "Permissions to new user directories" in the wiki, especially the last paragraph. When Dovecot doesn't know what permissions it should use, it defaults to 0700 to be safe. From CMarcus at Media-Brokers.com Mon Sep 17 16:40:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 09:40:09 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> Message-ID: <50572839.9000301@Media-Brokers.com> On 2012-09-17 8:45 AM, Timo Sirainen wrote: > Both CLucene and Lucene++ are C++ ports of the original Java Lucene. Hmmm... ok, I guess I'm missing something... Googling lucene++ leads me to this page: http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: http://lucene.apache.org/ and on that page it says (among other things): "/Lucene Core /, our flagship sub-project, provides Java-based indexing and search technology, as well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities." Which seems to say that it is java based...? -- Best regards, Charles From marcin at rootnode.net Mon Sep 17 16:40:35 2012 From: marcin at rootnode.net (=?UTF-8?Q?Marcin_H=C5=82ybin?=) Date: Mon, 17 Sep 2012 15:40:35 +0200 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 2012-09-17 15:30, Timo Sirainen wrote: > What's your mail_location setting? Note the "Permissions to new user > directories" in the wiki, especially the last paragraph. When Dovecot > doesn't know what permissions it should use, it defaults to 0700 to > be > safe. mail_location = maildir:~/Maildir Maybe that's the point. What do you propose? I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" -- M. From CMarcus at Media-Brokers.com Mon Sep 17 17:37:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:37:19 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50572839.9000301@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <5057359F.8070107@Media-Brokers.com> Ok, just to wrap up this thread for anyone who cares, I just got a reply from the clucene dev (in response to my asking him about the possible merger of clucene and lucene++) after he said he had fixed this bug and the next version would contain the fix: "More or less it's true. About a year ago we started to make Lucene++ to the new CLucene version, as Lucene++ (also written in C++) is a port of a newer Apache Lucene version (written in Java) as the one CLucene is a port of. But we did not want to simply merge them, but to adapt Lucene++ to the "design principles" of CLucene. E.g., Lucene++ makes heavy use of shared pointers. And in CLucene we wanted to reduce this usage in favor of performance. But this not finished and I cannot say when it will finished. Nevertheless, the new version of CLucene (if any) will be also C++ and not Java. Best regards, Veit" On 2012-09-17 9:40 AM, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache > Lucene/Solr page: > > http://lucene.apache.org/ > > and on that page it says (among other things): > > "/Lucene Core /, our flagship > sub-project, provides Java-based indexing and search technology, as > well as spellchecking, hit highlighting and advanced > analysis/tokenization capabilities." > > Which seems to say that it is java based...? > -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From tomas at blueboard.cz Mon Sep 17 17:40:55 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Mon, 17 Sep 2012 16:40:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes Message-ID: <50573677.7060108@blueboard.cz> Hello, I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 I`d like to have more secure configuration > one process to one connection. Thanks for help. Tomas My config: # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 6200 default_process_limit = 4096 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /var/log/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 1536 service_count = 1 } service imap { process_limit = 2048 } service pop3-login { service_count = 1 } service pop3 { process_limit = 2048 } shutdown_clients = no ssl_cert = References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: On 17.9.2012, at 16.40, Charles Marcus wrote: > On 2012-09-17 8:45 AM, Timo Sirainen wrote: >> Both CLucene and Lucene++ are C++ ports of the original Java Lucene. > > Hmmm... ok, I guess I'm missing something... > > Googling lucene++ leads me to this page: > > http://taschenorakel.de/mathias/2012/04/18/fulltext-search-benchmarks/ > > which has a link to 'lucene++', which takes me to the Apache Lucene/Solr page: > > http://lucene.apache.org/ This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus From tss at iki.fi Mon Sep 17 17:46:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:46:46 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573677.7060108@blueboard.cz> References: <50573677.7060108@blueboard.cz> Message-ID: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> On 17.9.2012, at 17.40, Tom?? Randa wrote: > Hello, > > I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). > Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. From tss at iki.fi Mon Sep 17 17:47:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 17:47:50 +0300 Subject: [Dovecot] Mailbox directory permissions (LDA, 0600 -> 0660, virtual users) In-Reply-To: References: <4267EF77-6B42-440A-AE95-2EFADBEA1500@iki.fi> <4A579ABB-6EA5-4F33-BD31-D3B8DB0972C2@iki.fi> Message-ID: On 17.9.2012, at 16.40, Marcin H?ybin wrote: > On 2012-09-17 15:30, Timo Sirainen wrote: >> What's your mail_location setting? Note the "Permissions to new user >> directories" in the wiki, especially the last paragraph. When Dovecot >> doesn't know what permissions it should use, it defaults to 0700 to be >> safe. > > mail_location = maildir:~/Maildir > > Maybe that's the point. What do you propose? > I don't think I can use %d and %n as mentioned in documentation because in mailbox path I have additional user_id directory. Home directory is retrieved from database with this part: "CONCAT_WS('/', home, u.uid, d.domain_name, u.user_name) AS home" Then your current possibilities are: a) mkdir + chmod the directory when a new user is created b) change Dovecot sources From CMarcus at Media-Brokers.com Mon Sep 17 17:52:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 17 Sep 2012 10:52:36 -0400 Subject: [Dovecot] clucene build warnings? In-Reply-To: References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> Message-ID: <50573934.1070909@Media-Brokers.com> On 2012-09-17 10:44 AM, Timo Sirainen wrote: > This is the only page I'm aware of for the project: > https://github.com/luceneplusplus/LucenePlusPlus Weird... wonder which is which, or if they are one and the same... Not much activity on that one either (most things are over a year old)... Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? -- Best regards, Charles From support at oeko.net Mon Sep 17 18:08:53 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 17:08:53 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota Message-ID: <20120917150852.GA13113@spruce.wiehl.oeko.net> Hi, I have now verified that there is a problem with quota (and sieve?). Summary: If the "maildirsize" file gets garbled (= every once in a while), mail is getting lost randomly. Details: I have a Debian box (6.0.5/amd64) with a Postfix server (2.7.1), which delivers email via Dovecot (1.2.15) to local disk (ext3). My email is filtered via a sieve script (wish I could have something better, eg. procmail). The account data is fetched from an SQL database, and I have a default quota set in dovecot.conf (200MB). My own mailbox has no quota (*:storage=0, mentioned in the database). A client sent an email to me, which is logged as being delivered like normal to my mailbox, but doesn't show up there. After poking around, I decided to simply delete the maildirsize file and have the client send that email again, and voila, it shows up in my mailbox. At the same time, however, other emails get delivered just fine, so the heinous thing is that you don't immediately notice that something fishy is going on - only when the client actually calls you (sort of defeats some uses of email)... The problem is worsened by the fact that you can't tell *which* emails get lost - you only end up with a random subset of "your" messages. The problem occurred now after running for a few months. FWIW, the problem with the garbled maildirsize file resulted in a different user's email getting bounced due to "mailbox full", when it wasn't. Questions: * Is this a known problem (Google was unhelpful/unwieldy)? * How can I prevent the problem, short of setting up a cron job to delete that file every week? * How can I log the delivery of messages in dovecot's log, not only the POP3/IMAP activity when people fetch their email? TIA! Kind regards, --Toni++ From tomas at blueboard.cz Mon Sep 17 18:10:55 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:10:55 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> Message-ID: <50573D7F.1040005@blueboard.cz> Hello, thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. Tomas R. Dne 17.09.12 16:46, Timo Sirainen napsal(a): > On 17.9.2012, at 17.40, Tom?? Randa wrote: > >> Hello, >> >> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 > That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. > From tss at iki.fi Mon Sep 17 18:16:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:16:20 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. On 17.9.2012, at 18.10, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >> > From tss at iki.fi Mon Sep 17 18:18:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:18:37 +0300 Subject: [Dovecot] clucene build warnings? In-Reply-To: <50573934.1070909@Media-Brokers.com> References: <5055E506.7040704@Media-Brokers.com> <46CF3364-77AC-42C6-B1FF-D37F1A9BC486@iki.fi> <50571AB8.3010505@Media-Brokers.com> <9102C2EF-FF02-4BBE-A9E2-6888031B7921@iki.fi> <50572839.9000301@Media-Brokers.com> <50573934.1070909@Media-Brokers.com> Message-ID: On 17.9.2012, at 17.52, Charles Marcus wrote: > On 2012-09-17 10:44 AM, Timo Sirainen wrote: >> This is the only page I'm aware of for the project: https://github.com/luceneplusplus/LucenePlusPlus > > Weird... wonder which is which, or if they are one and the same... > > Not much activity on that one either (most things are over a year old)... > > Timo, do you foresee any issues using clucene, now or the foreseeable future (year or more)? Some very large installations are successfully using it. I'm not planning on breaking it anytime soon. From f.bonnet at esiee.fr Mon Sep 17 18:21:05 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:21:05 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50573FE1.10906@esiee.fr> hello what is the result of the following command ? sysctl -a | grep kern.maxproc On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From f.bonnet at esiee.fr Mon Sep 17 18:22:16 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 17 Sep 2012 17:22:16 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <50573D7F.1040005@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <50574028.3070608@esiee.fr> and this one sysctl -a | grep kern.maxprocperuid On 09/17/2012 05:10 PM, Tom?? Randa wrote: > Hello, > > thanks for answer! I know that message come from kernel. But I have > set ulimit -u to 8192 for uid 89, so I think I forgot to increase some > value in dovecot.conf.. > > > Tomas R. > > > Dne 17.09.12 16:46, Timo Sirainen napsal(a): >> On 17.9.2012, at 17.40, Tom?? Randa wrote: >> >>> Hello, >>> >>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot >>> 2.1.8 version and cannot get over 1024 processes. I`ve increased >>> process_limit in imap and pop3 and still have messages in syslog >>> like: maxproc limit exceeded by uid 89, please see tuning(7) and >>> login.conf(5). >>> Did I forget to increase some other value? System values are higher >>> - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >> That message comes from the kernel. Dovecot doesn't use login.conf. >> You need to set ulimit -u manually (= in init.d script) before >> starting Dovecot. >> > -- CYBERDROID Inc. Frank BONNET 01.45.92.66.17 Service des Moyens Informatiques Generaux ESIEE PARIS Cit? Descartes / BP 99 93162 NOISY-LE-GRAND Cedex http://www.esiee.fr From tomas at blueboard.cz Mon Sep 17 18:24:24 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:24:24 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> Message-ID: <505740A8.3090803@blueboard.cz> On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) Thanks for your time Tomas R. Dne 17.09.12 17:16, Timo Sirainen napsal(a): > Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. > > On 17.9.2012, at 18.10, Tom?? Randa wrote: > >> Hello, >> >> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >> >> >> Tomas R. >> >> >> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>> >>>> Hello, >>>> >>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>> From tss at iki.fi Mon Sep 17 18:26:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:26:10 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <505740A8.3090803@blueboard.cz> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> Message-ID: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. On 17.9.2012, at 18.24, Tom?? Randa wrote: > On FreeBSD in standard behavior is basic user limit set by sysctl kern.maxprocperuid, which is standard to 5547 and I increased it to 8192. When I run under UID 89 limit command or ulimit -u I get answer: > > cputime unlimited > filesize unlimited > datasize 33554432 kbytes > stacksize 524288 kbytes > coredumpsize unlimited > memoryuse unlimited > vmemoryuse unlimited > descriptors 36864 > memorylocked unlimited > maxproc 8192 > sbsize unlimited > swapsize unlimited > > > Could It be some dovecot vs. freebsd related error? Or I "dont see" anything? :) > > > Thanks for your time > > Tomas R. > > > Dne 17.09.12 17:16, Timo Sirainen napsal(a): >> Where did you set the ulimit -u? It doesn't seem to be used by Dovecot processes. In any case there's nothing in dovecot.conf that can help you. >> >> On 17.9.2012, at 18.10, Tom?? Randa wrote: >> >>> Hello, >>> >>> thanks for answer! I know that message come from kernel. But I have set ulimit -u to 8192 for uid 89, so I think I forgot to increase some value in dovecot.conf.. >>> >>> >>> Tomas R. >>> >>> >>> Dne 17.09.12 16:46, Timo Sirainen napsal(a): >>>> On 17.9.2012, at 17.40, Tom?? Randa wrote: >>>> >>>>> Hello, >>>>> >>>>> I have configuration question. Running FreeBSD 9.0 / AMD64 / Dovecot 2.1.8 version and cannot get over 1024 processes. I`ve increased process_limit in imap and pop3 and still have messages in syslog like: maxproc limit exceeded by uid 89, please see tuning(7) and login.conf(5). >>>>> Did I forget to increase some other value? System values are higher - kern.maxprocperuid: 8192 and kern.maxproc: 12288 >>>> That message comes from the kernel. Dovecot doesn't use login.conf. You need to set ulimit -u manually (= in init.d script) before starting Dovecot. >>>> > From tss at iki.fi Mon Sep 17 18:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 17 Sep 2012 18:45:05 +0300 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> Message-ID: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> On 17.9.2012, at 18.26, Timo Sirainen wrote: > It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: 1. dovecot master process starts, inheriting the ulimits from the shell that starts it 2. Dovecot checks the current limit and increases it if necessary 3. Forked child processes inherit the parent's ulimits 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. kern.maxprocperuid I guess means the upper allowed limit for the ulimit. From tomas at blueboard.cz Mon Sep 17 18:58:12 2012 From: tomas at blueboard.cz (=?windows-1252?Q?Tom=E1=9A_Randa?=) Date: Mon, 17 Sep 2012 17:58:12 +0200 Subject: [Dovecot] Can`t get over 1024 processes In-Reply-To: <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> References: <50573677.7060108@blueboard.cz> <396FE5C2-0400-4DFD-98CA-69D9E916FB82@iki.fi> <50573D7F.1040005@blueboard.cz> <505740A8.3090803@blueboard.cz> <5DAB4CC9-B7A5-4DBE-866E-EC0AA69256CB@iki.fi> <0EE30790-BF30-4D6E-9625-8B32F7FB6CB9@iki.fi> Message-ID: <50574894.4090609@blueboard.cz> kern.maxprocperuid is maximum allowed user limit, But in standard it is set to maximum imho. And if dovecot checks limit why it did not increases it on FreeBSD? :( ? Hard limit 8192 on my box is much more. Dne 17.09.12 17:45, Timo Sirainen napsal(a): > On 17.9.2012, at 18.26, Timo Sirainen wrote: > >> It doesn't work like that with Dovecot. ulimit -u 8192 must be run just before dovecot binary is started. ulimits elsewhere are irrelevant. > > Although the strange thing is that Dovecot v2.1 is supposed to automatically increase this limit at startup. Maybe FreeBSD does work in a different way than Linux? The way Linux works (and Dovecot assumes things work) is: > > 1. dovecot master process starts, inheriting the ulimits from the shell that starts it > 2. Dovecot checks the current limit and increases it if necessary > 3. Forked child processes inherit the parent's ulimits > 4. setuid(X) fails if UID X has reached the ulimit value (according the current process's ulimit - not some other value read from somewhere) > > There is no reading of X's ulimit value anywhere, it's the same for all Dovecot child processes. limits.conf and such are read and applied by PAM, which is done before 1 step. > > kern.maxprocperuid I guess means the upper allowed limit for the ulimit. > From support at oeko.net Mon Sep 17 19:06:40 2012 From: support at oeko.net (Toni Mueller) Date: Mon, 17 Sep 2012 18:06:40 +0200 Subject: [Dovecot] dovecot 1.x: problem with quota In-Reply-To: <20120917150852.GA13113@spruce.wiehl.oeko.net> References: <20120917150852.GA13113@spruce.wiehl.oeko.net> Message-ID: <20120917160640.GA16928@spruce.wiehl.oeko.net> Hi, On Mon, Sep 17, 2012 at 05:08:53PM +0200, Toni Mueller wrote: > I have now verified that there is a problem with quota (and sieve?). I was probably jumping to conclusions, as the delivery problem happens again - statistically extremely unlikely after this short while. kind regards, --Toni++ From kgc at corp.sonic.net Tue Sep 18 02:02:29 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Mon, 17 Sep 2012 16:02:29 -0700 Subject: [Dovecot] Proxy connection timeouts Message-ID: <5057AC05.2050904@corp.sonic.net> We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) I'm curious if anyone else has seen any similar problems or has any suggestions. # dovecot -n # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 Scientific Linux release 6.3 (Carbon) auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes auth_verbose_passwords = sha1 auth_worker_max_count = 64 mail_fsync = always mail_log_prefix = "%s(%u): session=%{session} " mail_plugins = stats zlib maildir_very_dirty_syncs = yes mmap_disable = yes passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = imap driver = pam } plugin { lazy_expunge = DELETED_MESSAGES. stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 service anvil { client_limit = 10000 } service auth { client_limit = 10000 vsz_limit = 512 M } service doveadm { inet_listener { port = 1842 } unix_listener doveadm-server { mode = 0666 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 7000 process_min_avail = 32 } service imap-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-imap-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 4096 vsz_limit = 512 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 2000 process_min_avail = 32 } service pop3-postlogin { executable = script-login -d /etc/dovecot/bin/sonic-pop3-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin process_limit = 4096 } service stats { fifo_listener stats-mail { mode = 0666 } } shutdown_clients = no ssl = required ssl_ca = References: <50449193.8080101@um.es> Message-ID: <50581BCC.7050607@um.es> El 11/09/12 16:24, Timo Sirainen escribi?: > On 3.9.2012, at 14.16, Angel L. Mateo wrote: > >> Moreover... according to previous mails (thread http://www.dovecot.org/list/dovecot/2012-June/066691.html) there is a patch to remove a specific user entry from cache. This patch I think is included in dovecot 2.1.9 (which I'm running) and the syntax is (I think): >> >> doveadm auth cache flush >> 4 cache entries flushed > > Yep. > >> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) > > Works in my tests. > Is this cache the same than the user information cache? The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: doveadm auth cache flush in the logs I get: Sep 18 08:47:13 myotis34 dovecot: auth: Debug: master in: CACHE-FLUSH#0111#011 now I ask for user information with: doveadm user and this is what I get from logs: Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:19 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:19 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:19 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master in: USER#0111#011#011service=doveadm Sep 18 08:47:41 myotis34 dovecot: auth: Debug: prefetch(): passdb didn't return userdb entries, trying the next userdb Sep 18 08:47:41 myotis34 dovecot: auth: Debug: userdb-cache(): hit: #011home=/home/otros/99/151299#011uid=151299#011gid=405 Sep 18 08:47:41 myotis34 dovecot: auth: Debug: master out: USER#0111#011#011home=/home/otros/99/151299#011uid=151299#011gid=405 As you can see in the third message, it is still using information from userdb cache From alessio at skye.it Tue Sep 18 10:50:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 18 Sep 2012 09:50:30 +0200 Subject: [Dovecot] Shared Mailboxes in a multi domain environment In-Reply-To: <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> References: <5056CC95.8030209@skye.it> <1F745A08-D0FD-42A2-9D0E-5F5AF6FE3480@iki.fi> Message-ID: <505827C6.6080204@skye.it> Il 17/09/2012 14:44, Timo Sirainen ha scritto: >> So, why don't provide a way to restrict shared mailboxes also for dict in SQL? >> >> One way could be to add to "user_shares" table a column "domains" which is the same domain of the mailbox'owner and a config option, for acl, like acl_only_for_same_domain =yes/no so dovecot can add a "WHERE %n = domain" for the SELECT. > If you don't include %d as part of the shared namespace prefix you already restrict the users sharing within same domain, no need for extra SQL WHEREs. Good, after change my shared namespace from namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } to namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ ## <<=- change %u to %n separator = / subscriptions = no type = shared } shared mailboxes have become available only inside the same domains. Thanks! -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From ap00 at mail.ru Tue Sep 18 14:04:31 2012 From: ap00 at mail.ru (=?UTF-8?B?0JDQu9C10LrRgdC10Lkg0J/QsNC90LrRgNCw0YLQvtCy?=) Date: Tue, 18 Sep 2012 15:04:31 +0400 Subject: [Dovecot] =?utf-8?q?lda_bug=3A_multiple_passdb_ignored?= Message-ID: <1347966271.760993075@f26.mail.ru> Greetings. When dovecot configured to use multiple passdb and static userdb as: #password database in multiple ldap subtree passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-real.conf } passdb { ?? driver = ldap ?? args = /usr/local/etc/dovecot/dv-ldap-virtual.conf } userdb { ?driver = static ?args = uid=pop gid=mailnull } all works fine but... When LDA deliver mail it check passdb but only first appeared in configuration: 18.09.2012 15:00:18 auth: Debug: master in: USER??????? 1?????? info at zzz.zz ???? service=lda 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword 18.09.2012 15:00:18 lda: Debug: auth input: 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND?? 1 1 P.S. dovecot version 2.1.7 sysytem x86 FreeBSD 9.0 From janfrode at tanso.net Tue Sep 18 14:38:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 18 Sep 2012 13:38:53 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? Message-ID: We have a sieve script doing sieve_before to sort spam to spam-folders. Now I'm trying to configure the "Trash" plugin, but it doesn't seem to work.. I noticed my config file says: # Space separated list of plugins to load (none known to be useful so far). Do NOT # try to load IMAP plugins here. #mail_plugins = and that doveconf doesn't list any plugins loaded for "protocol sieve". Should we load quota and trash here ? -jf From Jost.Krieger+dovecot at rub.de Tue Sep 18 14:40:12 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 18 Sep 2012 13:40:12 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) Message-ID: <20120918114011.GH8536@ruhr-uni-bochum.de> It seems there is no way to correct te quota limit downward for some mailboxes, like: quota_rule9 = UCE-TMP:storage=-10%% The idea is to not eat up the users storage with spam, so that spam stays in the queue (for some time) while real mail gets delivered. There *is* a line } else if (*value != '-' && relative_rule) { in quota.c, but nothing else handles it, starting with quota_rule_parse_percentage. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From stephan at rename-it.nl Tue Sep 18 16:41:16 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 15:41:16 +0200 Subject: [Dovecot] trash plugin together with sieve_before ? In-Reply-To: References: Message-ID: <505879FC.4010907@rename-it.nl> On 9/18/2012 1:38 PM, Jan-Frode Myklebust wrote: > We have a sieve script doing sieve_before to sort spam to > spam-folders. Now I'm trying to configure the "Trash" plugin, but it > doesn't seem to work.. I noticed my config file says: > > # Space separated list of plugins to load (none known to be useful > so far). Do NOT > # try to load IMAP plugins here. > #mail_plugins = > > and that doveconf doesn't list any plugins loaded for "protocol > sieve". Should we load quota and trash here ? Keep in mind that the Sieve language and the Sieve protocol are two different things. The Sieve language is what processes incoming e-mails; for Dovecot it is executed by a plugin for LTMP or LDA during message delivery. The Sieve protocol - which is actually called ManageSieve - is used to upload Sieve scripts to the Server. That mail_plugins setting applies to the Sieve protocol, for which currently no useful plugins are known. The Sieve protocol service does not (currently) open any mail stores, which makes configuring mail-related plugins for it - such as quota and trash - utterly useless. So, in short, the answer to the above question is no. To help you further, we need some more information about your configuration. First of all, please provide the output of dovecot -n to this list. Also, describe more thoroughly what is going wrong. Do you see any log messages, etc. Regards, Stephan. From stephan at rename-it.nl Tue Sep 18 17:34:51 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 16:34:51 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058868B.8080904@rename-it.nl> Hello Dovecot users, I've been busy recently with some wholly unrelated Dovecot IMAP patches, so Sieve development moved a bit into the background. This release therefore consists mostly of small bugfixes. The most important change is the fix of a bug in ManageSieve's CHECKSCRIPT command that causes a segfault. Quite a few people have encountered this problem recently, so it is about time I release this now. Changelog v0.3.2: + sieve-refilter tool: improved man page documentation by explicitly specifying the syntax used for mailbox arguments. + Sieve: spamtest and virustest extensions: improved trace debugging of score calculation. + Sieve: made error messages about exceeding the maximum number of actions more verbose. - Sieve tools: fixed problems with running as root: sievec and sieve-dump now ignore mail_uid and mail_gid settings when run as root. - Sieve: fixed bug in action accounting (for limit checking): increase action instance count only when an action is actually created. - Sieve: include extension: fixed namespace separation of :global and :personal scripts. - ManageSieve: fixed segfault bug triggered by CHECKSCRIPT command. - Fixed linking with ld.gold. - Fixed several Clang compile warnings and a few potential bugs. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.2.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Tue Sep 18 18:48:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 18:48:19 +0300 Subject: [Dovecot] Mailbox aliases In-Reply-To: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> References: <90E6B18F-5B3A-4623-9B7E-CF9C2D8C5550@iki.fi> Message-ID: This is now implemented as a plugin in hg. Usage in the commit message: http://hg.dovecot.org/dovecot-2.1/rev/f5bb9f6b304d Also there's a generic change where symlinks within same directory are treated as mailbox aliases, which aren't counted towards quota. On 2.9.2012, at 18.32, Timo Sirainen wrote: > I've been wondering about ways to handle all the various different Sent/Sent Messages/Sent Items/etc mailboxes that different clients create and if there could be a way to make them work at least somewhat better. I'm mainly thinking about ISP-like installations where users are using tons of different clients and there's no way to get everyone configured properly. Of course the eventual solution is hopefully that all the clients will support SPECIAL-USE extension, but while waiting for that maybe mailbox aliases could help some. > > So one possibility would be to force autocreation of wanted specific mailbox names, and create aliases for the other commonly used mailboxes. The main difference here for the current behavior is that all of the mailboxes would contain the same mails, instead of different mails going to different mailboxes with different clients. So for example: > > .. > mailbox Sent { > auto = create > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > alias_for = Sent > } > .. > > If "Sent Messages" doesn't exist: > - DELETE Sent works as usual > - RENAME Sent works as usual > > If "Sent Messages" is created: > - it's symlinked to Sent > - DELETE "Sent messages" deletes the symlink, not the contents > - RENAME "Sent messages" isn't allowed > - DELETE/RENAME Sent fails with message: Delete/Rename for Sent not allowed before "Sent Messages" is deleted. > > Thoughts? > > The main disadvantage is that clients would still see two different mailboxes, and some would download contents from both of them. > > I was thinking about creating some wiki page which could list the best working configurations and also a client list, which would specify if it supports SPECIAL-USE and what the default mailbox names it uses are. From tss at iki.fi Tue Sep 18 19:04:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:04:06 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > passdb { > driver = ldap > args = /usr/local/etc/dovecot/dv-ldap-virtual.conf > } Does dv-ldap-virtual.conf have auth_bind=yes? From tss at iki.fi Tue Sep 18 19:17:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:17:50 +0300 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120918114011.GH8536@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: On 18.9.2012, at 14.40, Jost Krieger wrote: > It seems there is no way to correct te quota limit downward for some > mailboxes, like: > > quota_rule9 = UCE-TMP:storage=-10%% > > The idea is to not eat up the users storage with spam, so that spam > stays in the queue (for some time) while real mail gets delivered. Interesting idea. This was pretty easy to fix: http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. From tss at iki.fi Tue Sep 18 19:31:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:31:08 +0300 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50581BCC.7050607@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> Message-ID: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >> >> Works in my tests. >> > Is this cache the same than the user information cache? Yes. > The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: > > doveadm auth cache flush What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: doveadm auth cache flush foo # make sure it isn't there doveadm user foo doveadm auth cache flush foo Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. From tss at iki.fi Tue Sep 18 19:34:49 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:34:49 +0300 Subject: [Dovecot] Proxy connection timeouts In-Reply-To: <5057AC05.2050904@corp.sonic.net> References: <5057AC05.2050904@corp.sonic.net> Message-ID: <15C8389A-3C3A-461B-8FF8-FBACBDDB6A6C@iki.fi> On 18.9.2012, at 2.02, Kelsey Cummings wrote: > We are seeing a few (0-15) proxy failures like the following out of ~3m successful proxied connections a day. Average session creation load over our peak hour is about 47/sec. The backend servers aren't logging anything that would suggest any internal problem like insufficient processes to handle the load. It doesn't seem to happen when utilization is lowest at night. > > dovecot: imap-login: Error: proxy(foo): connect(1.1.1.1, 143) failed: Connection timed out (after 63 secs) > > I'm curious if anyone else has seen any similar problems or has any suggestions. I once had similar problems when the proxy backend was Courier. The problems went away after migration to Dovecot was complete. The possibilities are either: a) The backend server is busy and doesn't have a chance to accept() the connection. b) Packets get dropped in the network and the retry packet is slow in coming (or also gets lost). Changing some kernel settings might help with a). There are also kernel settings that specify how SYN resend is attempted, you could try reducing it to a few seconds. From ap00 at mail.ru Tue Sep 18 19:40:39 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 20:40:39 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> References: <1347966271.760993075@f26.mail.ru> <93041EBA-FF8A-404A-9EFA-7BDA232750CE@iki.fi> Message-ID: <106-722288046.20120918204039@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:04:06 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> passdb { >> driver = ldap >> args = /usr/local/etc/dovecot/dv-ldap-virtual.conf >> } TS> Does dv-ldap-virtual.conf have auth_bind=yes? No. All ldap connection configured to use one prescribed ldap binding. P.S. Thank you for your work. -- Best regards, Anthony mailto:ap00 at mail.ru From tss at iki.fi Tue Sep 18 19:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 19:58:53 +0300 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <1347966271.760993075@f26.mail.ru> References: <1347966271.760993075@f26.mail.ru> Message-ID: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> On 18.9.2012, at 14.04, ??????? ????????? wrote: > When dovecot configured to use multiple passdb and static userdb as: .. > all works fine but... > > When LDA deliver mail it check passdb but only first appeared in configuration: > > 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda > 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss > 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword > 18.09.2012 15:00:18 lda: Debug: auth input: > 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: > 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 From robert at schetterer.org Tue Sep 18 20:47:09 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 18 Sep 2012 19:47:09 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <5058B39D.5030501@schetterer.org> Am 18.09.2012 18:17, schrieb Timo Sirainen: > On 18.9.2012, at 14.40, Jost Krieger wrote: > >> It seems there is no way to correct te quota limit downward for some >> mailboxes, like: >> >> quota_rule9 = UCE-TMP:storage=-10%% >> >> The idea is to not eat up the users storage with spam, so that spam >> stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf > > Now, the next problem is that if LDA can't save the message to UCE-TMP because of quota failure it saves it to INBOX. I'm not really sure how that should be fixed, since generally it's a good idea to do it.. > some allready existing way is i.e with milter and postfix tagged spam get in hold queue for i.e human admin investigate, or some quarantaine with amavis for sure there are benefits getting dovecot involved in spam tagging but i would leave spam tagging to the mailserver in first way and deliver it to Junk folder by global sieve rule with giving more quota or ignoring Quota to Junk folder, but i see this isnt the same what you might wanna goal -- Best Regards MfG Robert Schetterer From jcblanco at fi.upm.es Tue Sep 18 21:18:21 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 18 Sep 2012 20:18:21 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 Message-ID: <5058BAED.5080505@fi.upm.es> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the new pigeonhole version, however I've got the follwing errors when compiling: gcc -shared .libs/rfc2822.o .libs/edit-mail.o .libs/sieve-settings.o .libs/sieve-message.o .libs/sieve-smtp.o .libs/sieve-lexer.o .libs/sieve-script.o .libs/sieve-script-file.o .libs/sieve-script-dict.o .libs/sieve-ast.o .libs/sieve-binary.o .libs/sieve-binary-file.o .libs/sieve-binary-code.o .libs/sieve-binary-debug.o .libs/sieve-parser.o .libs/sieve-address.o .libs/sieve-validator.o .libs/sieve-generator.o .libs/sieve-interpreter.o .libs/sieve-runtime-trace.o .libs/sieve-code-dumper.o .libs/sieve-binary-dumper.o .libs/sieve-result.o .libs/sieve-error.o .libs/sieve-objects.o .libs/sieve-stringlist.o .libs/sieve-comparators.o .libs/sieve-match-types.o .libs/sieve-address-parts.o .libs/sieve-match.o .libs/sieve-commands.o .libs/sieve-code.o .libs/sieve-actions.o .libs/sieve-extensions.o .libs/sieve-plugins.o .libs/cmp-i-octet.o .libs/cmp-i-ascii-casemap.o .libs/mcht-is.o .libs/mcht-contains.o .libs/mcht-matches.o .libs/tst-truefalse.o .libs/tst-not.o .libs/tst-anyof.o .libs/tst-allof.o .libs/tst-address.o .libs/tst-header.o .libs/tst-exists.o .libs/tst-size.o .libs/cmd-require.o .libs/cmd-stop.o .libs/cmd-if.o .libs/cmd-keep.o .libs/cmd-redirect.o .libs/cmd-discard.o .libs/ext-fileinto.o .libs/ext-reject.o .libs/ext-envelope.o .libs/ext-encoded-character.o .libs/sieve.o -Wl,--whole-archive ../../src/lib-sieve/plugins/vacation/.libs/libsieve_ext_vacation.a ../../src/lib-sieve/plugins/subaddress/.libs/libsieve_ext_subaddress.a ../../src/lib-sieve/plugins/comparator-i-ascii-numeric/.libs/libsieve_ext_comparator-i-ascii-numeric.a ../../src/lib-sieve/plugins/relational/.libs/libsieve_ext_relational.a ../../src/lib-sieve/plugins/regex/.libs/libsieve_ext_regex.a ../../src/lib-sieve/plugins/copy/.libs/libsieve_ext_copy.a ../../src/lib-sieve/plugins/imap4flags/.libs/libsieve_ext_imap4flags.a ../../src/lib-sieve/plugins/include/.libs/libsieve_ext_include.a ../../src/lib-sieve/plugins/body/.libs/libsieve_ext_body.a ../../src/lib-sieve/plugins/variables/.libs/libsieve_ext_variables.a ../../src/lib-sieve/plugins/enotify/.libs/libsieve_ext_enotify.a ../../src/lib-sieve/plugins/notify/.libs/libsieve_ext_notify.a ../../src/lib-sieve/plugins/environment/.libs/libsieve_ext_environment.a ../../src/lib-sieve/plugins/mailbox/.libs/libsieve_ext_mailbox.a ../../src/lib-sieve/plugins/date/.libs/libsieve_ext_date.a ../../src/lib-sieve/plugins/spamvirustest/.libs/libsieve_ext_spamvirustest.a ../../src/lib-sieve/plugins/ihave/.libs/libsieve_ext_ihave.a ../../src/lib-sieve/plugins/editheader/.libs/libsieve_ext_editheader.a ../../src/lib-sieve/plugins/vnd.dovecot/debug/.libs/libsieve_ext_debug.a ../../src/lib-sieve/plugins/vnd.dovecot/duplicate/.libs/libsieve_ext_duplicate.a -Wl,--no-whole-archive -Wl,--rpath -Wl,/usr/lib64/dovecot -L/var/tmp/dovecot-2.1-pigeonhole-0.3.2-219_3-root/usr/lib64/dovecot -L/usr/lib64/dovecot -ldovecot-lda -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-storage/.libs -L/usr/kerberos/lib64 -L/home/jc/rpmbuild/BUILD/dovecot-2.1.9/src/lib-dovecot/.libs -lrt -m64 -mtune=generic -Wl,-soname -Wl,libdovecot-sieve.so.0 -o .libs/libdovecot-sieve.so.0.0.0 /usr/bin/ld: cannot find -ldovecot-lda collect2: ld returned 1 exit status libtool: install: error: relink `libdovecot-sieve.la' with the above command before installing it make[4]: *** [install-dovecot_pkglibLTLIBRARIES] Error 1 make[4]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src/lib-sieve' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/home/jc/rpmbuild/BUILD/dovecot-2.1-pigeonhole-0.3.2/src' make: *** [install-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.50565 (%install) It seems that the error is related to the ld.gold fix, If I revert the c52a0c561311 patch the error is not produced Regards From stephan at rename-it.nl Tue Sep 18 21:49:19 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 20:49:19 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058BAED.5080505@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> Message-ID: <5058C22F.1070704@rename-it.nl> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: > Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the > new pigeonhole version, however I've got the follwing errors when > compiling: > > It seems that the error is related to the ld.gold fix, If I revert the > c52a0c561311 patch the error is not produced Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, but I forgot the Dovecot v2.1 tree. Fixed now: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c This only happens when you compile against installed Dovecot headers, people using --with-dovecot= (like me) would not have seen this. I'll push out a new release once 2.1.10 comes out. I'm being told that is imminent. Regards, Stephan. From tss at iki.fi Tue Sep 18 21:52:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 21:52:27 +0300 Subject: [Dovecot] v2.1.10 released Message-ID: <1347994347.13571.67.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig The director bugfix is pretty important at least when running in two director setups, since it can break the directors until both of them are manually shut down. I'm not entirely sure if the same happens in >2 director setups, but I'd think so (although I do wonder why I haven't seen it happen in them). BTW. I'm hoping for v2.2.alpha1 release within a week or two and to get it stabilized rather quickly. + imap: Implemented THREAD=ORDEREDSUBJECT extension. + Added "doveadm exec" command to easily execute commands from libexec_dir, e.g. "doveadm exec imap -u user at domain" + Added "doveadm copy" command. + doveadm copy/move: Added optional user parameter to specify the source username. This allows easily copying mails between different users. + Added namespace { disabled } setting to quickly enable/disable namespaces. This is especially useful when its value is returned by userdb. + Added mailbox_alias plugin. It allows creating mailbox aliases using symlinks. + imapc storage: Added imapc_max_idle_time setting to force activity on connection. + fts-solr: Expunging multiple messages is now faster. - director: In some conditions director may have disconnected from another director (without logging about it), thinking it was sending invalid data. - imap: Various fixes to listing mailboxes. - pop3-migration plugin: Avoid disconnection from POP3 server due to idling. - login processes crashed if there were a lot of local {} or remote {} settings blocks. From ap00 at mail.ru Tue Sep 18 22:04:42 2012 From: ap00 at mail.ru (Anthony Pankov) Date: Tue, 18 Sep 2012 23:04:42 +0400 Subject: [Dovecot] lda bug: multiple passdb ignored In-Reply-To: <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> References: <1347966271.760993075@f26.mail.ru> <59A7A611-404F-482A-96B3-6B1A4A085181@iki.fi> Message-ID: <186-713645281.20120918230442@mail.ru> Hello Timo, Tuesday, September 18, 2012, 8:58:53 PM, you wrote: TS> On 18.9.2012, at 14.04, ??????? ????????? wrote: >> When dovecot configured to use multiple passdb and static userdb as: TS> .. >> all works fine but... >> >> When LDA deliver mail it check passdb but only first appeared in configuration: >> >> 18.09.2012 15:00:18 auth: Debug: master in: USER 1 info at zzz.zz service=lda >> 18.09.2012 15:00:18 auth: Debug: cache(info at zzz.zz): miss >> 18.09.2012 15:00:18 auth: Debug: ldap(info at zzz.zz): pass search: base=ou=account,ou=it,dc=planar,dc=local scope=subtree filter=(&(objectClass=x-mailUser)(uid=info)) fields=x-mailPassword >> 18.09.2012 15:00:18 lda: Debug: auth input: >> 18.09.2012 15:00:18 auth: Debug: cache(info at planar.spb.ru): hit: >> 18.09.2012 15:00:18 auth: Debug: master out: NOTFOUND 1 1 TS> This was an interesting bug. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/c38060f0ad58 Thank you very much. -- Best regards, Anthony mailto:ap00 at mail.ru From stephan at rename-it.nl Tue Sep 18 22:36:34 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 18 Sep 2012 21:36:34 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.3 for Dovecot v2.1.10 Message-ID: <5058CD42.3000507@rename-it.nl> Hello Dovecot users, Version 0.3.2 was a bit broken, because it failed to compile against installed Dovecot headers. Changelog v0.3.3: - Fixed compile against installed Dovecot headers. This was broken by the ld.gold fix in the previous release. The release is available as follows: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.3.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From interfasys at gmail.com Tue Sep 18 23:32:18 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 21:32:18 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) Message-ID: <5058DA52.7070400@gmail.com> Hello, There is a big problem with 2.1.10. Here is what I saw in the logs: Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed with signal 6 (core dumped) Reverting to 2.1.9 with the exact same config file works just fine Cheers, Olivier From Ralf.Hildebrandt at charite.de Tue Sep 18 23:34:03 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 18 Sep 2012 22:34:03 +0200 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <20120918203403.GK15878@charite.de> * interfaSys s?rl : > Hello, > > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine So far I'm not having problems wit 2.1.10 here (coming from 2.1.9) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Sep 18 23:42:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 18 Sep 2012 23:42:57 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058DA52.7070400@gmail.com> References: <5058DA52.7070400@gmail.com> Message-ID: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> On 18.9.2012, at 23.32, interfaSys s?rl wrote: > There is a big problem with 2.1.10. > > Here is what I saw in the logs: > Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) > Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed > with signal 6 (core dumped) > > Reverting to 2.1.9 with the exact same config file works just fine gdb backtrace? http://dovecot.org/bugreport.html From interfasys at gmail.com Wed Sep 19 00:18:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Tue, 18 Sep 2012 22:18:07 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> Message-ID: <5058E50F.10000@gmail.com> I thought it would be easily identifiable and a common problem, but apparently not :) Here is the trace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at user.c:17 _module_tmp = (void *) 0x10a84d28 asu = (struct antispam_user *) 0x10a84d28 tmp = 0x0 #9 0x00000000104f1f04 in hook_mail_user_created () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #10 0x00000000104efaf9 in mail_user_init () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000010495858 in mail_storage_service_init_post.46586.6405 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000010495582 in mail_storage_service_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #13 0x00000000104951b3 in mail_storage_service_lookup_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #14 0x000000000041f112 in client_create_from_input (input=0x7fffffffe940, login_client=0x10a550c0, fd_in=11, fd_out=11, input_buf=0x7fffffffe900, error_r=0x7fffffffe938) at main.c:202 user = (struct mail_storage_service_user *) 0x10a55141 mail_user = (struct mail_user *) 0x3 client = (struct client *) 0x7fffffffec70 set = (const struct imap_settings *) 0x7fffffffe900 flags = 32767 #15 0x000000000041eeaa in login_client_connected (client=0x10a550c0, username=0x10a04093 "test at example.com", extra_fields=0x10a04118) at main.c:272 input = {module = 0x423d0f "imap", service = 0x423d0f "imap", username = 0x10a04093 "test at example.com", session_id = 0x10a55120 "UCFMXQDK5gBewxZf", local_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "N.8Q", '\0' , __u6_addr16 = {11854, 20792, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1362636366, 0, 0, 0}}}, ip4 = {s_addr = 1362636366}}}, remote_ip = {family = 2, u = {ip6 = {__u6_addr = {__u6_addr8 = "^?\026_", '\0' , __u6_addr16 = {50014, 24342, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1595327326, 0, 0, 0}}}, ip4 = {s_addr = 1595327326}}}, local_port = 0, remote_port = 0, userdb_fields = 0x10a04118, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} error = (const __unknown__ *) 0x0 input_buf = {data = 0x10a55141, used = 3, priv = {0x0, 0x0, 0x3, 0x0, 0x0}} #16 0x00000000105a229f in master_login_auth_finish.11302 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x00000000105efa50 in master_login_auth_callback.11253.2275 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #18 0x0000000010596f56 in master_login_auth_input_user.11520 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x000000001058bf99 in master_login_auth_input.11492 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x00000000105e9b34 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x00000000105d1ba7 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #22 0x00000000105d1a55 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x00000000105d1790 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #24 0x000000000041ed5d in main (argc=1, argv=0x7fffffffec60) at main.c:389 set_roots = {0x424020, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ed9d , failure_callback = 0x4194ee } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x424020, 0x0} Cheers, Olivier On 18/09/2012 21:42, Timo Sirainen wrote: > On 18.9.2012, at 23.32, interfaSys s?rl wrote: > >> There is a big problem with 2.1.10. >> >> Here is what I saw in the logs: >> Sep 18 22:22:23 imap(): Panic: file buffer.c: line 86 >> (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) >> Sep 18 22:22:23 imap(): Fatal: master: service(imap): child 26833 killed >> with signal 6 (core dumped) >> >> Reverting to 2.1.9 with the exact same config file works just fine > > gdb backtrace? http://dovecot.org/bugreport.html > > > From tss at iki.fi Wed Sep 19 02:42:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 02:42:34 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <1CB6DE70-A784-4D90-8127-BF87EDA84407@iki.fi> On 19.9.2012, at 0.18, interfaSys s?rl wrote: > I thought it would be easily identifiable and a common problem, but > apparently not :) .. > #8 0x00000000109c770b in antispam_user_created (user=0x10a83040) at > user.c:17 Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. From robert at schetterer.org Wed Sep 19 10:40:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 09:40:29 +0200 Subject: [Dovecot] v2.1.10 released In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <505976ED.4090602@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > + Added mailbox_alias plugin. It allows creating mailbox aliases using > symlinks. Hi Timo thx for coding, no Problems so far, after update, but mailbox_alias plugin should be described at http://wiki2.dovecot.org/Plugins/ -- Best Regards MfG Robert Schetterer From bangkokmaco at gmail.com Wed Sep 19 11:00:55 2012 From: bangkokmaco at gmail.com (cc "maco" young) Date: Wed, 19 Sep 2012 15:00:55 +0700 Subject: [Dovecot] ssl cert for mail server Message-ID: for testing a new ssl cert. it works ok for browsers, but > openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 => verify error:num=19:self signed certificate in certificate chain is this ssl cert - as it's constructed - is ok for mail clients? (realize needs to be on mail port etc - right now talking about the cert itself.) have had problems with thunderbird, and was wondering if this might be part of the problem. From amateo at um.es Wed Sep 19 11:22:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 10:22:42 +0200 Subject: [Dovecot] Deleting a folder with & character Message-ID: <505980D2.3060401@um.es> Hello, One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). I can see the maildir with a doveadm list command: amateo_adm at myotis31:~$ sudo doveadm mailbox list -u ... INBOX.Kron & SPM ... but I can't delete it, neithe rename it: amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? From acrow at integrafin.co.uk Wed Sep 19 11:29:12 2012 From: acrow at integrafin.co.uk (Alex Crow) Date: Wed, 19 Sep 2012 09:29:12 +0100 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <50598258.1030005@integrafin.co.uk> On 19/09/12 09:22, Angel L. Mateo wrote: > Hello, > > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe > created a long time ago, when we use courier imap as pop/imap server, > but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox > doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u > 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox > doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a > specific syntax or escape to indicate the name? > Hi, I see something similar when I try to doveadm import maildir folders (to mdbox) with ampersands - I get the same "Mailbox doesn't exist: INBOX.Foo &- Bar" with that extraneous hyphen. Cheers Alex -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. "Transact" is operated by Integrated Financial Arrangements plc Domain House, 5-7 Singer Street, London EC2A 4BQ Tel: (020) 7608 4900 Fax: (020) 7608 5300 (Registered office: as above; Registered in England and Wales under number: 3727592) Authorised and regulated by the Financial Services Authority (entered on the FSA Register; number: 190856) From tss at iki.fi Wed Sep 19 12:14:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 12:14:07 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <505980D2.3060401@um.es> References: <505980D2.3060401@um.es> Message-ID: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> On 19.9.2012, at 11.22, Angel L. Mateo wrote: > One of my users has a mailbox named 'INBOX.Kron & SPM' (maybe created a long time ago, when we use courier imap as pop/imap server, but I'm not sure). > > I can see the maildir with a doveadm list command: > > amateo_adm at myotis31:~$ sudo doveadm mailbox list -u > ... > INBOX.Kron & SPM > ... > > but I can't delete it, neithe rename it: > > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron & SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron & SPM: Mailbox doesn't exist: INBOX.Kron &- SPM > amateo_adm at myotis31:~$ sudo doveadm mailbox delete -u 'INBOX.Kron \& SPM' > doveadm(jrfv): Error: Can't delete mailbox INBOX.Kron \& SPM: Mailbox doesn't exist: INBOX.Kron \&- SPM > > is there any way to delete it with doveadm command? should I use a specific syntax or escape to indicate the name? It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). From amateo at um.es Wed Sep 19 12:15:53 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 11:15:53 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> Message-ID: <50598D49.8060507@um.es> El 18/09/12 18:31, Timo Sirainen escribi?: > On 18.9.2012, at 9.59, Angel L. Mateo wrote: > >>>> So I'm running this command. Whenever I run it, I get the message that 3 (sometimes, is 4) entries are removed, but user information isn't really reloaded and I doubt it is really removed from cache (I have the user in a passwd-file and information used by imap processes is still the old one, no the new one, changed before the flush) >>> >>> Works in my tests. >>> >> Is this cache the same than the user information cache? > > Yes. > >> The parameter of the user I want to change is his quota, so I have modified quota value in my ldap diretory, then I run: >> >> doveadm auth cache flush > > What is your doveconf -n output and the dovecot-ldap.conf contents? Is with or without @domain? Also try this: > > doveadm auth cache flush foo # make sure it isn't there > doveadm user foo > doveadm auth cache flush foo > > Does the second flush return 1 or 0 entries? If 0, then there's a problem. If 1, then it really should have worked. > > You could try also if disabling userdb prefetch makes any difference. And if you still have multiple userdb try with only one. > I have made the test in my test server (it has no real activity). In this server, user entry is refreshed correctly. But the same test in my production servers fails. I have checked (in the production one) that the second flush delete entries (in fact, 2, not 1): amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush angel.luis 2 cache entries flushed amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis userdb: angel.luis home : /home/alumnos/46/113246 uid : 113246 gid : 1001 quota_rule: *:storage=400M but quota information is not reloaded from ldap server. I have also checked my ldap server, and dovecot is not performing any search operation for the user after flushing him from the cache. I have attached my doveconf -n. In my ldap configuration I have: user_attrs = irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule pass_attrs = irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule and I have check that my test and production server has the same configuration (but client_limit and number of processes). Any idea? From c at roessner-network-solutions.com Wed Sep 19 12:26:33 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 11:26:33 +0200 Subject: [Dovecot] OT: z-Push Message-ID: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Hi, just a short question: I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. Also I can not remove that folder over ActiveSync. Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 Thanks in advance -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From robert at schetterer.org Wed Sep 19 12:42:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 11:42:12 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: <50599374.4010902@schetterer.org> Am 19.09.2012 11:26, schrieb Christian R??ner: > Hi, > > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? yes it works with dovecot > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 sorry tested , there is no option for create folder on my android, need test account offlist? > > Thanks in advance > > -Christian R??ner > > --- > Bachelor of Science Informatik > Erlenwiese 14, 36304 Alsfeld > T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 > USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com > -- Best Regards MfG Robert Schetterer From CMarcus at Media-Brokers.com Wed Sep 19 13:36:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 19 Sep 2012 06:36:12 -0400 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> Message-ID: <5059A01C.2080706@Media-Brokers.com> On 2012-09-19 5:14 AM, Timo Sirainen wrote: > It's an invalid mailbox name and you can't access it using doveadm. > Either rename it with "mv" or via IMAP protocol (not entirely sure if > it works via IMAP either). Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? Thanks Timo, -- Best regards, Charles From alessio at skye.it Wed Sep 19 13:54:33 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 12:54:33 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <5059A469.6060604@skye.it> Hi, I have found this strange problem. I'm working with Debian 6, dovecot 2.1.9 and vpopmail-auth. LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. If disable ACL (acl and imap_acl from mail_plugins) in dovecot.conf "Segmentation fault" go away. Moreover, without ACL at the first delivery folders like Sent, Drafts, Trash, Spam appear immediately. An example, new user: root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? maildirsize ??? new ??? tmp root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Segmentation fault root at demo-vpop ~ # echo $? 139 root at demo-vpop ~ # tree /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ -a /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/ ??? cur ??? dovecot-acl-list ??? dovecot.index.cache ??? dovecot.index.log ??? dovecot-uidlist ??? dovecot-uidvalidity ??? dovecot-uidvalidity.50599a7f ??? maildirsize ??? new ? ??? 1348049535.M21758P19264.demo-vpop.cbsolt.net,S=2025,W=2071 ??? tmp dovecot log (with ulimit -c unlimited before starting Dovecot): Sep 19 12:06:50 auth: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:06:50 auth-worker(18728): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:06:50 auth-worker(18728): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:06:50 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 dict: Info: mysql(localhost): Connected to database dovecot lda log: Sep 19 12:06:50 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:06:50 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:06:50 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:06:50 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:06:50 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found now works fine: root at demo-vpop ~ # cat testmail.txt | /usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi2 at qboxdns.it Sep 19 12:08:18 auth: Debug: master in: USER 1 cecchi2 at qboxdns.it service=lda Sep 19 12:08:18 auth-worker(18763): Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot/auth Sep 19 12:08:18 auth-worker(18763): Debug: vpopmail(cecchi2 at qboxdns.it): lookup user=cecchi2 domain=qboxdns.it Sep 19 12:08:18 auth: Debug: master out: USER 1 cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 dict: Info: mysql(localhost): Connected to database dovecot Sep 19 12:08:18 lda: Debug: Loading modules from directory: /usr/local/dovecot-2.1/lib/dovecot Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Sep 19 12:08:18 lda: Debug: Module loaded: /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Sep 19 12:08:18 lda: Debug: auth input: cecchi2 at qboxdns.it uid=89 gid=89 home=/home/vpopmail/domains/qboxdns.it/cecchi2 quota_rule=*:backend=10485760S Sep 19 12:08:18 lda: Debug: Added userdb setting: plugin/quota_rule=*:backend=10485760S Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Effective uid=89, gid=89, home=/home/vpopmail/domains/qboxdns.it/cecchi2 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=* bytes=10485760 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota rule: root=UserQuota mailbox=Trash bytes=+104857600 messages=0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=cecchi2 at qboxdns.it, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: maildir++: root=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, index=, control=, inbox=/home/vpopmail/domains/qboxdns.it/cecchi2/Maildir, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/Maildir:INDEX=~/Maildir/shared/%u Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: shared: root=/usr/local/dovecot-2.1/var/run/dovecot, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: initializing backend with data: vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: acl username = cecchi2 at qboxdns.it Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl: owner = 0 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: Global ACL directory: /usr/local/dovecot-2.1/etc/dovecot/global-acls Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=UserQuota backend=maildir args= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Quota root: name=User quota backend=dict args=:noenforcing:proxy::quota Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: dict quota: user=raw mail user, uri=proxy::quota, noenforcing=1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: none: root=, index=, control=, inbox=, alt= Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Destination address: cecchi2 at qboxdns.it (source: user at hostname) Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Info: msgid=<5059979E.3080402 at cecchi.biz>: saved mail to INBOX Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/INBOX not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Drafts not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Drafts doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Spam not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Spam doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Trash not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Trash doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls/Sent not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent/dovecot-acl not found Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir/.Sent doesn't exist yet, using default permissions Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: Namespace : Using permissions from /home/vpopmail/domains/qboxdns.it/cecchi2/Maildir: mode=0700 gid=-1 Sep 19 12:08:18 lda(cecchi2 at qboxdns.it): Debug: acl vfile: file /usr/local/dovecot-2.1/etc/dovecot/global-acls//.DEFAULT not found # 2.1.9: /usr/local/dovecot-2.1/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_size = 512 k auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_worker_max_count = 60 default_login_user = nobody dict { acl = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-share-folder.conf quota = mysql:/usr/local/dovecot-2.1/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota acl maildir_very_dirty_syncs = yes mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = cache_key=%s%u webmail=192.168.113.215 driver = vpopmail } plugin { acl = vfile:/usr/local/dovecot-2.1/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl quota = maildir:UserQuota quota2 = dict:User quota::noenforcing:proxy::quota quota_rule2 = Trash:storage=+100M } protocols = imap pop3 sendmail_path = /var/qmail/bin/sendmail service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { service_count = 0 } service pop3-login { service_count = 0 } ssl_cert = http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From florob at babelmonkeys.de Wed Sep 19 14:56:26 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 13:56:26 +0200 Subject: [Dovecot] ssl cert for mail server In-Reply-To: References: Message-ID: <5059B2EA.3040605@babelmonkeys.de> Am 19.09.2012 10:00, schrieb cc "maco" young: > for testing a new ssl cert. it works ok for browsers, but > >> openssl s_client -crlf -connect ms1.trailsandtribulations.net:443 > > => verify error:num=19:self signed certificate in certificate chain > > is this ssl cert - as it's constructed - is ok for mail clients? (realize > needs to be on mail port etc - right now talking about the cert itself.) > have had problems with thunderbird, and was wondering if this might be > part of the problem. > Hi, first of all this is likely off topic for this ML, I'll still answer though, since I'm always intrigued by TLS problems. The reason openssl doesn't accept this cert, while your browser does, is quite likely that your system wide accepted CAs don't include Starfield Technologies, while your browser's CAs do (This is the case for Firefox and Thunderbird). However, I suspect that your mail addresses are of the form , and ms1.trailsandtribulations.net is what is in your MX record. As such the certificate needs to be valid for trailsandtribulations.net, and not ms1.trailsandtribulations.net. So you either need trailsandtribulations.net as your CN, or a SAN of type DNSName for trailsandtribulations.net. Cf. https://tools.ietf.org/html/rfc6125 for best practices on generating certificates. Regards, Florian From ml at cibest.com Wed Sep 19 15:15:48 2012 From: ml at cibest.com (Cibest ML) Date: Wed, 19 Sep 2012 14:15:48 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <50504ED3.5030601@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> Message-ID: <5059B774.6060808@cibest.com> On 12/09/2012 10:58, Cibest ML wrote: > On 11/09/2012 20:48, Timo Sirainen wrote: >> On 8.9.2012, at 18.07, Cibest ML wrote: >> >>> - Message delivered by LMTP >>> Quota name Type Value Limit >>> User quota STORAGE 2653 524288 >>> User quota MESSAGE 1 - >>> >>> - quota recalc >>> Quota name Type Value Limit >>> User quota STORAGE 5307 524288 >>> User quota MESSAGE 2 - >> Quota recalculation is clearly seeing the message twice. My first >> guess was that you had two namespaces pointing to the same location, >> but you seem to have only one. Do you have symlinks in your Maildir? >> > Hi, > Nothing specific on the namespace side. > The only entry is the default one in 10-mail.conf > namespace inbox { > inbox = yes > } > > I tried to comment it out but it doesn't change anything > > And no symlinks in the Maildir. > Here's the file structure of Maildir for the test mailbox > > ./dovecot-uidvalidity.504df4e7 > ./dovecot-uidvalidity > ./.Sent > ./.Sent/maildirfolder > ./.Sent/new > ./.Sent/dovecot.index.log > ./.Sent/tmp > ./.Sent/dovecot-uidlist > ./.Sent/cur > ./.Spam > ./.Spam/maildirfolder > ./.Spam/new > ./.Spam/dovecot.index.log > ./.Spam/tmp > ./.Spam/dovecot-uidlist > ./.Spam/cur > ./new > ./subscriptions > ./.Drafts > ./.Drafts/maildirfolder > ./.Drafts/new > ./.Drafts/dovecot.index.log > ./.Drafts/tmp > ./.Drafts/dovecot-uidlist > ./.Drafts/cur > ./.Drafts/dovecot-keywords > ./dovecot.index.log > ./tmp > ./dovecot.mailbox.log > ./dovecot-uidlist > ./.Trash > ./.Trash/maildirfolder > ./.Trash/new > ./.Trash/dovecot.index.log > ./.Trash/tmp > ./.Trash/dovecot-uidlist > ./.Trash/cur > ./cur > ./cur/1347286291.M194531P4329.mspooll01,S=3748,W=3847:2,Sa > ./dovecot-keywords > ./dovecot.index.cache > > -- > Regards > S?bastien We did some more investigations as we had to enable the no enforcing quota option to prevent too many compltains from our customers. As the issue seems to be linked to the inbox folder, we tried to focus on that. According to Timo's last post, that could be linked to namespace or symlinks (we don't have any). There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). Now doing some check via telnet doesn't show any mysterious "Inbox" list "" "*" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Spam" * LIST (\HasNoChildren) "." "Drafts" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "INBOX" lsub "" "*" * LSUB () "." "Trash" * LSUB () "." "Sent" * LSUB () "." "Spam" * LSUB () "." "Drafts" * LSUB () "." "Inbox" 04 select INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. 04 select Inbox * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1346832313] UIDs valid * OK [UIDNEXT 82] Predicted next UID * OK [HIGHESTMODSEQ 51] Highest 04 OK [READ-WRITE] Select completed. It's working for both but I checked on other working dovecot servers, that seems to be a normal behavior for Inbox folder I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) -- S?bastien From tss at iki.fi Wed Sep 19 15:42:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:42:16 +0300 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <5059A01C.2080706@Media-Brokers.com> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> Message-ID: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> On 19.9.2012, at 13.36, Charles Marcus wrote: > On 2012-09-19 5:14 AM, Timo Sirainen wrote: >> It's an invalid mailbox name and you can't access it using doveadm. Either rename it with "mv" or via IMAP protocol (not entirely sure if it works via IMAP either). > > Just for clarification... it is invalid because of the '&' character? Is there a list of dovecot (or is this simply the IMAP protocol in general?) considers invalid characters in folder names somewhere? mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. Dovecot has also some additional restrictions on characters (mainly control chars), but they are only enforced when creating new mailboxes. Opening existing mailboxes with control chars is allowed. From tss at iki.fi Wed Sep 19 15:47:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:47:28 +0300 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <5059B774.6060808@cibest.com> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> Message-ID: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> On 19.9.2012, at 15.15, Cibest ML wrote: > There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). No idea. INBOX is treated case-insensitively by IMAP protocol. > I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. From tss at iki.fi Wed Sep 19 15:48:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 15:48:27 +0300 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059A469.6060604@skye.it> References: <5059A469.6060604@skye.it> Message-ID: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> On 19.9.2012, at 13.54, Alessio Cecchi wrote: > LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:52:41 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:52:41 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: References: <20120918114011.GH8536@ruhr-uni-bochum.de> Message-ID: <20120919125241.GF2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:17:50 PM GMT, Timo Sirainen wrote: > > On 18.9.2012, at 14.40, Jost Krieger wrote: > ... > > The idea is to not eat up the users storage with spam, so that spam > > stays in the queue (for some time) while real mail gets delivered. > > Interesting idea. This was pretty easy to fix: > > http://hg.dovecot.org/dovecot-2.1/rev/74d639b2a5bf Thank you! > Now, the next problem is that if LDA can't save the message to UCE-TMP > because of quota failure it saves it to INBOX. I'm not really sure how > that should be fixed, since generally it's a good idea to do it.. Oops, that's not what I want! How about another command line parameter for lda to suppress the emergency save? We know exactly when we are trying to save probable spam. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Jost.Krieger+dovecot at rub.de Wed Sep 19 15:53:39 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 14:53:39 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5058B39D.5030501@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> Message-ID: <20120919125339.GG2010@ruhr-uni-bochum.de> On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > some allready existing way is i.e > with milter and postfix tagged spam get in hold queue for i.e human > admin investigate, or some quarantaine with amavis Human admin is not a good idea if you have 50000 users :-) Quarantine's a possibility, but that's taking it out of "Mail" and adds complexity. > for sure there are benefits getting dovecot involved in spam tagging > but i would leave spam tagging to the mailserver in first way and > deliver it to Junk folder by global sieve rule with giving more quota or > ignoring Quota to Junk folder, but i see this isnt the same what you > might wanna goal That's just what we want to do, *but* although it's in the Wiki I don't think it a good idea to *raise* the limit for the junk folder if you *deliver* to it. It's somewhat ok if you move mail there and someone picks it up (we do that with a different folder). But on delivery it means at some point you can receive spam but no real messages. Not nice. "ignore" may work if it does what I think (never even count bytes and messages in this folder) but it is open to abuse ... Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From amateo at um.es Wed Sep 19 16:00:45 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 19 Sep 2012 15:00:45 +0200 Subject: [Dovecot] Syntax for doveadm auth cache In-Reply-To: <50598D49.8060507@um.es> References: <50449193.8080101@um.es> <50581BCC.7050607@um.es> <905DCFFA-9AE0-4773-BFA0-1A42EABEDFFB@iki.fi> <50598D49.8060507@um.es> Message-ID: <5059C1FD.1040702@um.es> (I forgot the attach) El 19/09/12 11:15, Angel L. Mateo escribi?: > El 18/09/12 18:31, Timo Sirainen escribi?: >> On 18.9.2012, at 9.59, Angel L. Mateo wrote: >> >>>>> So I'm running this command. Whenever I run it, I get the >>>>> message that 3 (sometimes, is 4) entries are removed, but user >>>>> information isn't really reloaded and I doubt it is really removed >>>>> from cache (I have the user in a passwd-file and information used >>>>> by imap processes is still the old one, no the new one, changed >>>>> before the flush) >>>> >>>> Works in my tests. >>>> >>> Is this cache the same than the user information cache? >> >> Yes. >> >>> The parameter of the user I want to change is his quota, so I >>> have modified quota value in my ldap diretory, then I run: >>> >>> doveadm auth cache flush >> >> What is your doveconf -n output and the dovecot-ldap.conf contents? Is >> with or without @domain? Also try this: >> >> doveadm auth cache flush foo # make sure it isn't there >> doveadm user foo >> doveadm auth cache flush foo >> >> Does the second flush return 1 or 0 entries? If 0, then there's a >> problem. If 1, then it really should have worked. >> >> You could try also if disabling userdb prefetch makes any difference. >> And if you still have multiple userdb try with only one. >> > > I have made the test in my test server (it has no real activity). > In this server, user entry is refreshed correctly. But the same test in > my production servers fails. > > I have checked (in the production one) that the second flush delete > entries (in fact, 2, not 1): > > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm auth cache flush > angel.luis > 2 cache entries flushed > amateo_adm at myotis31:/etc/dovecot/conf.d$ sudo doveadm user angel.luis > userdb: angel.luis > home : /home/alumnos/46/113246 > uid : 113246 > gid : 1001 > quota_rule: *:storage=400M > > but quota information is not reloaded from ldap server. > > I have also checked my ldap server, and dovecot is not performing > any search operation for the user after flushing him from the cache. > > I have attached my doveconf -n. In my ldap configuration I have: > user_attrs = > irisMailbox=mail,homeDirectory=home,uidNumber=uid,gidNumber=gid,quota=quota_rule > > pass_attrs = > irisMailbox=userdb_mail,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,quota=userdb_quota_rule > > > and I have check that my test and production server has the same > configuration (but client_limit and number of processes). > > Any idea? > -------------- next part -------------- # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.19um1 x86_64 Ubuntu 12.04.1 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1024 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_plugins = quota mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = BORRADOS. quota = dict:User quota::file:%h/Maildir/dovecot.quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+1G sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es protocols = imap pop3 lmtp sieve service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From alessio at skye.it Wed Sep 19 16:03:58 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:03:58 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> Message-ID: <5059C2BE.7050006@skye.it> Il 19/09/2012 14:48, Timo Sirainen ha scritto: > On 19.9.2012, at 13.54, Alessio Cecchi wrote: > >> LDA is configured and works fine but the problem is when the first message arrive "dovecot-lda" return a "Segmentation fault", the message is written to the user's Mailbox but the message remains, also, in the queue of qmail (deferral: Segmentation_fault/) and at the second attempt is delivered fine. > gdb backtrace would be very helpful in figuring out the problem: http://dovecot.org/bugreport.html > Obvious ... :-) Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver -d cecchi10 at qboxdns.it'. Program terminated with signal 11, Segmentation fault. #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 221 if (dict->dict == NULL) is enough? Thanks -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Wed Sep 19 16:07:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 19 Sep 2012 15:07:31 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C2BE.7050006@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> Message-ID: <5059C393.5050209@skye.it> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: > Il 19/09/2012 14:48, Timo Sirainen ha scritto: >> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >> >>> LDA is configured and works fine but the problem is when the first >>> message arrive "dovecot-lda" return a "Segmentation fault", the >>> message is written to the user's Mailbox but the message remains, >>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>> the second attempt is delivered fine. >> gdb backtrace would be very helpful in figuring out the problem: >> http://dovecot.org/bugreport.html >> > > Obvious ... :-) > > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so...done. > Loaded symbols for /usr/local/dovecot-2.1/lib/dovecot/lib01_acl_plugin.so > Reading symbols from > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so...done. > Loaded symbols for > /usr/local/dovecot-2.1/lib/dovecot/lib10_quota_plugin.so > Core was generated by `/usr/local/dovecot-2.1/libexec/dovecot/deliver > -d cecchi10 at qboxdns.it'. > Program terminated with signal 11, Segmentation fault. > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > 221 if (dict->dict == NULL) > > is enough? This is the full bt: (gdb) bt full #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 ns = ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = 0x0, v_modifiable = 0x0} ids = 0x1928658 i = dest = ret = -883075307 #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( backend=0x1944240) at acl-backend-vfile-acllist.c:297 auser = 0x1949a08 iter = 0x0 acllist_path = 0x1928658 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" ret = ns = 0x1943e50 output = 0x0 st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = 33152, st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1348059559, tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, 0, 0}} path = 0x1928210 file_mode = 384 dir_mode = 448 gid = 4294967295 list = info = rootdir = 0x1928610 "Sent" origin = 0x194d178 "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" fd = 8 #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) at acl-backend-vfile-acllist.c:311 acllist_path = #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh (backend=0x1944240) at acl-backend-vfile-acllist.c:153 __FUNCTION__ = "acl_backend_vfile_acllist_refresh" #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 acllist = #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( _aclobj=0x19444e0) at acl-backend-vfile.c:858 old_validity = validity = {global_validity = {last_check = 0, last_read_time = 1348059559, last_mtime = 0, last_size = 0}, local_validity = {last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}, mailbox_validity = { last_check = 0, last_read_time = 0, last_mtime = 0, last_size = 0}} mtime = 0 ret = 26515976 #6 0x00007f2fc9fc125e in acl_backend_get_default_rights (backend=0x1944240, mask_r=0x28) at acl-backend.c:164 No locals. #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:107 alist = nonowner_list_ctx = ret = backend = 0x1944240 acl_mask = 0x1 ns = 0x1943e50 update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = 0x7f2fcbf2ba88, glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, update_only = 0, match_parents = 0} name = #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 _data_stack_cur_id = 2 ctx = 0x1946b20 pool = i = inboxcase = #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple (list=0x194cc00, patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at mailbox-list-iter.c:158 ctx = ret = __FUNCTION__ = "mailbox_list_iter_init_multiple" #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, pattern=, flags=1348059559) at mailbox-list-iter.c:58 patterns = {0x7f2fc9db76dc "*", 0x0} #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:73 ctx = 0x7f2fcb5beef3 info = #12 quota_count (root=0x1944950, bytes_r=, count_r=0x7fff362dfff0) at quota-count.c:111 i = 0 ret = 0 #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, value_r=0x7fff362e0038) at quota-dict.c:113 dt = bytes = 0 count = 0 #14 0x00007f2fc9db395a in dict_quota_update_callback ( ret=, context=0x1949a08) at quota-dict.c:178 value = 1 #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, line_r=) at dict-client.c:265 ctx = 0x1984190 #16 client_dict_read_one_line (dict=0x194c7f0, line_r=) at dict-client.c:356 id = 1 line = ret = 0 __FUNCTION__ = "client_dict_read_one_line" #17 0x00007f2fcb592565 in client_dict_wait (_dict=) at dict-client.c:520 dict = 0x194c7f0 line = 0x0 ret = #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) at quota-dict.c:90 root = 0x1944950 #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 pool = 0x194c5c0 #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 quota = 0x1944720 i = 2 #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) at quota-storage.c:412 quser = 0x1949a30 quota_set = 0x1946f30 #22 0x00007f2fcb88188e in mail_user_unref (_user=) at mail-user.c:153 user = 0x19483c0 __FUNCTION__ = "mail_user_unref" #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 set_roots = {0x604640, 0x0} ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, src_envelope_sender = 0x0, dest_user = 0x0, dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, var_expand_table = 0x192e050, tried_default_save = true, saved_mail = true, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = user = 0x192d3c2 "cecchi10 at qboxdns.it" errstr = 0x0 path = 0x7fff362e0368 "\351\a" storage_service = 0x192f3a0 service_user = 0x192fd48 service_input = {module = 0x4034d5 "lda", service = 0x4034d5 "lda", username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = 0x1943f30 user_source = destaddr_source = 0x403594 "user at hostname" process_euid = stderr_rejection = false ret = c = error = MAIL_ERROR_NONE (gdb) Sorry -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From robert at schetterer.org Wed Sep 19 17:03:45 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:03:45 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125339.GG2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> Message-ID: <5059D0C1.5050202@schetterer.org> Am 19.09.2012 14:53, schrieb Jost Krieger: > On Tue 18 Sep 2012 07:47:09 PM GMT, Robert Schetterer wrote: > >> some allready existing way is i.e >> with milter and postfix tagged spam get in hold queue for i.e human >> admin investigate, or some quarantaine with amavis > > Human admin is not a good idea if you have 50000 users :-) > Quarantine's a possibility, but that's taking it out of "Mail" and adds > complexity. as ever ,this is different in different places specially Quarantine is wide used > >> for sure there are benefits getting dovecot involved in spam tagging >> but i would leave spam tagging to the mailserver in first way and >> deliver it to Junk folder by global sieve rule with giving more quota or >> ignoring Quota to Junk folder, but i see this isnt the same what you >> might wanna goal > > That's just what we want to do, *but* although it's in the Wiki I don't > think it a good idea to *raise* the limit for the junk folder if you > *deliver* to it. It's somewhat ok if you move mail there and someone > picks it up (we do that with a different folder). But on delivery it > means at some point you can receive spam but no real messages. Not nice. > > "ignore" may work if it does what I think (never even count bytes and > messages in this folder) but it is open to abuse ... there is no good or evil to this case, its a design question which may change in different places,, the abuse case might be acceptable, as the Junk folder i.e is imap special used autoconfigured and has auto empty feature, i give 50 mb plus quota on Junk folder and delete mail older then 3 month there > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 19 17:25:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:25:05 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <1347994347.13571.67.camel@hurina> References: <1347994347.13571.67.camel@hurina> Message-ID: <5059D5C1.500@schetterer.org> Am 18.09.2012 20:52, schrieb Timo Sirainen: > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig > Hi Timo, i see a few rare errors with lucene and/or indexer since update /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out lucene-indexes) failed: Directory not empty Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) failed: Directory not empty lucene-indexes: IndexReader::open() failed (#1): File does not exist Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual method called Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called without an active exception Error: indexer failed to index mailbox INBOX should i care -- Best Regards MfG Robert Schetterer From Jost.Krieger+dovecot at rub.de Wed Sep 19 17:25:30 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 19 Sep 2012 16:25:30 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <5059D0C1.5050202@schetterer.org> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> Message-ID: <20120919142529.GI2010@ruhr-uni-bochum.de> On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > as ever ,this is different in different places > specially Quarantine is wide used Of course, it's often useful. > there is no good or evil to this case, its a design question which may > change in different places,, the abuse case might be acceptable, as the > Junk folder i.e is imap special used autoconfigured > and has auto empty feature, i give 50 mb plus quota on Junk folder > and delete mail older then 3 month there The abuse may be tolerable, but I just don't get how your +50M works. For me it would mean that someone with a large Junk folder just wouldn't get an more mail except something delivered straight to Junk. And if you don't deliver to Junk but the clients move stuff there, there's still a problem. I'm justing raising the limit for folders that clients use for deleting (like Trash or whatever it is called). Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From robert at schetterer.org Wed Sep 19 17:44:42 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 19 Sep 2012 16:44:42 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919142529.GI2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <5058B39D.5030501@schetterer.org> <20120919125339.GG2010@ruhr-uni-bochum.de> <5059D0C1.5050202@schetterer.org> <20120919142529.GI2010@ruhr-uni-bochum.de> Message-ID: <5059DA5A.1070908@schetterer.org> Am 19.09.2012 16:25, schrieb Jost Krieger: > On Wed 19 Sep 2012 04:03:45 PM GMT, Robert Schetterer wrote: > >> as ever ,this is different in different places >> specially Quarantine is wide used > > Of course, it's often useful. > >> there is no good or evil to this case, its a design question which may >> change in different places,, the abuse case might be acceptable, as the >> Junk folder i.e is imap special used autoconfigured >> and has auto empty feature, i give 50 mb plus quota on Junk folder >> and delete mail older then 3 month there > > The abuse may be tolerable, but I just don't get how your +50M works. the situation isnt really comparable, the plus is more configured for copy by human to i.e Junk folder, or i.e a client filter does this auto it makes most sense with trash, so user should mostly able to copy to trash, even if they have got full quota, this should avoid some errors in clients http://wiki2.dovecot.org/Quota/Configuration ... quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = SPAM:ignore ... > For me it would mean that someone with a large Junk folder just wouldn't > get an more mail except something delivered straight to Junk. And if you > don't deliver to Junk but the clients move stuff there, with auto loosing mail after some time in my setup which is the anounced policy, Trash and Junk are "autodeleted" folders no mail there ,rests forever *g there's still a > problem. I'm justing raising the limit for folders that clients use for > deleting (like Trash or whatever it is called). as i said its a design question, your policy is ok too no flame ,having your idea as alternative is always fine to have > > Yours > Jost Krieger > -- Best Regards MfG Robert Schetterer From interfasys at gmail.com Wed Sep 19 17:55:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 15:55:33 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5058E50F.10000@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> Message-ID: <5059DCE5.8030903@gmail.com> > Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. You are correct, it had not been compiled in a while. Here is a new backtrace: (gdb) bt full #0 0x0000000010709ebc in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000010708e43 in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000105b8e65 in default_fatal_finish.14395 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000010588e3d in i_internal_fatal_handler.14399 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x00000000105e4941 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x00000000105e7c77 in buffer_check_limits.13467 () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x00000000105e7a6a in buffer_write () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00000000105ce9f9 in array_idx_set_i () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at user.c:96 _module_tmp = (void *) 0x10a84d18 asu = (struct antispam_user *) 0x10a84d18 tmp = Variable "tmp" is not available. Cheers, Olivier From tss at iki.fi Wed Sep 19 17:59:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 17:59:38 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <5059DCE5.8030903@gmail.com> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> Message-ID: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > You are correct, it had not been compiled in a while. Here is a new > backtrace: .. > #6 0x00000000105e7a6a in buffer_write () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #7 0x00000000105ce9f9 in array_idx_set_i () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > user.c:96 > _module_tmp = (void *) 0x10a84d18 > asu = (struct antispam_user *) 0x10a84d18 > tmp = Variable "tmp" is not available. This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? From tss at iki.fi Wed Sep 19 18:02:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:02:07 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> Message-ID: <1348066927.13571.69.camel@hurina> On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: > On 19.9.2012, at 17.55, interfaSys s?rl wrote: > > >> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. > > > > You are correct, it had not been compiled in a while. Here is a new > > backtrace: > .. > > #6 0x00000000105e7a6a in buffer_write () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #7 0x00000000105ce9f9 in array_idx_set_i () from > > /usr/lib/dovecot/libdovecot.so.0 > > No symbol table info available. > > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at > > user.c:96 > > _module_tmp = (void *) 0x10a84d18 > > asu = (struct antispam_user *) 0x10a84d18 > > tmp = Variable "tmp" is not available. > > This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? Attached a helpful antispam patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 444 bytes Desc: not available URL: From interfasys at gmail.com Wed Sep 19 18:12:01 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 16:12:01 +0100 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: <5059E0C1.9070906@gmail.com> You are once again correct. The update script had an error and thus the plugin was not recompiled for 2.1.10... Thank you! Olivier On 19/09/2012 16:02, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >>>> Did you recompile antispam plugin? Is it the latest antispam? This line doesn't seem to match the current code. >>> >>> You are correct, it had not been compiled in a while. Here is a new >>> backtrace: >> .. >>> #6 0x00000000105e7a6a in buffer_write () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #7 0x00000000105ce9f9 in array_idx_set_i () from >>> /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >>> user.c:96 >>> _module_tmp = (void *) 0x10a84d18 >>> asu = (struct antispam_user *) 0x10a84d18 >>> tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? > > Attached a helpful antispam patch. > From tss at iki.fi Wed Sep 19 18:16:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 18:16:58 +0300 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <5059D5C1.500@schetterer.org> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> Message-ID: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> On 19.9.2012, at 17.25, Robert Schetterer wrote: > Am 18.09.2012 20:52, schrieb Timo Sirainen: >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >> > > Hi Timo, i see a few rare errors with lucene and/or indexer > since update Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out > lucene-indexes) failed: Directory not empty > > Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) > failed: Directory not empty > > lucene-indexes: IndexReader::open() failed (#1): File does not exist The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual > method called > Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called > without an active exception > > Error: indexer failed to index mailbox INBOX > > should i care > -- > Best Regards > MfG Robert Schetterer > From eugene at raptor.kiev.ua Wed Sep 19 19:13:13 2012 From: eugene at raptor.kiev.ua (Eugene Paskevich) Date: Wed, 19 Sep 2012 19:13:13 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: <1348066927.13571.69.camel@hurina> References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On Wed, 19 Sep 2012 18:02:07 +0300, Timo Sirainen wrote: > On Wed, 2012-09-19 at 17:59 +0300, Timo Sirainen wrote: >> On 19.9.2012, at 17.55, interfaSys s?rl wrote: >> >> >> Did you recompile antispam plugin? Is it the latest antispam? This >> line doesn't seem to match the current code. >> > >> > You are correct, it had not been compiled in a while. Here is a new >> > backtrace: >> .. >> > #6 0x00000000105e7a6a in buffer_write () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #7 0x00000000105ce9f9 in array_idx_set_i () from >> > /usr/lib/dovecot/libdovecot.so.0 >> > No symbol table info available. >> > #8 0x0000000010e2a1b5 in antispam_user_created (user=0x10a83040) at >> > user.c:96 >> > _module_tmp = (void *) 0x10a84d18 >> > asu = (struct antispam_user *) 0x10a84d18 >> > tmp = Variable "tmp" is not available. >> >> This still looks some kind of a version mismatch. Did you compile >> antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is >> also really from v2.1.10? > > Attached a helpful antispam patch. Erm... Am I missing something? http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c line 26, or actually line 32 given http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 -- Eugene Paskevich | *==)----------- | Plug me into eugene at raptor.kiev.ua | -----------(==* | The Matrix From jcblanco at fi.upm.es Wed Sep 19 19:22:29 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Wed, 19 Sep 2012 18:22:29 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5058C22F.1070704@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> Message-ID: <5059F145.1030803@fi.upm.es> On 18/09/2012 20:49, Stephan Bosch wrote: > On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >> Hi Stephan, I'm traying to rebuild the RH (Centos) EL5 RPM with the >> new pigeonhole version, however I've got the follwing errors when >> compiling: >> >> It seems that the error is related to the ld.gold fix, If I revert the >> c52a0c561311 patch the error is not produced > > Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, > but I forgot the Dovecot v2.1 tree. Fixed now: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > This only happens when you compile against installed Dovecot headers, > people using --with-dovecot= (like me) would not have seen this. > > I'll push out a new release once 2.1.10 comes out. I'm being told that > is imminent. Sorry, but using the new 0.3.3 version I'm getting the same error and I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in configure, not the installed headers. If needed I can send you the configure invocation command. Regards Juan C. Blanco > > Regards, > > Stephan. > > From tss at iki.fi Wed Sep 19 19:24:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 19 Sep 2012 19:24:22 +0300 Subject: [Dovecot] 2.1.10 crashes at login (buf->used <= buf->alloc) In-Reply-To: References: <5058DA52.7070400@gmail.com> <3E8273F3-C141-4482-8821-F9236DA6921A@iki.fi> <5058E50F.10000@gmail.com> <5059DCE5.8030903@gmail.com> <1771807C-2243-4BE3-AA5E-FA9EA2777E62@iki.fi> <1348066927.13571.69.camel@hurina> Message-ID: On 19.9.2012, at 19.13, Eugene Paskevich wrote: >>> This still looks some kind of a version mismatch. Did you compile antispam against v2.1.10 headers? /usr/lib/dovecot/libdovecot.so.0 is also really from v2.1.10? >> >> Attached a helpful antispam patch. > > Erm... Am I missing something? > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/7f94cc6b4d8e/src/antispam-plugin.c > line 26, or actually line 32 given > http://hg.dovecot.org/dovecot-antispam-plugin/annotate/e902fcd40d1d/antispam-plugin.c#l32 Oh, didn't notice it there below :) Wonder why it then didn't complain about version mismatch. From alec at alec.pl Wed Sep 19 20:10:16 2012 From: alec at alec.pl (A.L.E.C) Date: Wed, 19 Sep 2012 19:10:16 +0200 Subject: [Dovecot] Deleting a folder with & character In-Reply-To: <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> References: <505980D2.3060401@um.es> <013E7781-8013-4214-BFCD-1473CD669FB6@iki.fi> <5059A01C.2080706@Media-Brokers.com> <3EB95906-7EBF-4CDC-9148-893B11F9B21E@iki.fi> Message-ID: <5059FC78.7090100@alec.pl> On 09/19/2012 02:42 PM, Timo Sirainen wrote: > mUTF7 uses the & character for encoding non-ASCII data. The & character is encoded in mUTF7 as "&-". Dovecot nowadays uses UTF8 internally for all mailbox names, and it's simply not possible to create from UTF8 string a valid mUTF7 string that contains & alone. I'm confused. I have no problems with creating and using a folder with utf8 name containing '&' character. So, what problem do you describe? doveadm is working with utf8 right? So, why it can't delete such folder. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From florob at babelmonkeys.de Wed Sep 19 20:43:09 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 19:43:09 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> Message-ID: <505A042D.5090705@babelmonkeys.de> Am 11.09.2012 20:21, schrieb Timo Sirainen: > On 11.9.2012, at 21.07, Florian Zeitz wrote: > >>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>> >> Well, either that, or we could use a union of all known context structs >> there. > > That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. > >> Or we could use i_malloc() and i_free() under the assumption hmac_init() >> + hmac_final() calls are always matched. > > Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. > Here is an updated version of the patch. It uses the variant defining the largest allowed content size. I choose 1024 bit (128 Byte), since that is what most of the SHA-3 candidates use, so it's hopefully somewhat future proof. It's also rebased to the current head and switches the hmac uses added since the last version over to the changed API. It also fixes an "interesting" compiler warning I got with clang, which complained about the fact that struct hmac_context* was (re)declared in the prototype of hmac_md5_get_cram_context() and therefore separate from the one declared in hmac.h. Regards, Florian -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 0aa0a656176c9ca1c3f187999cd4725bf52665e5 # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE (1024 / 8) + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From interfasys at gmail.com Wed Sep 19 20:52:20 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Wed, 19 Sep 2012 18:52:20 +0100 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message Message-ID: <505A0654.3010501@gmail.com> I see the same crashes on FreeBSD. Turning off ACL makes the crashes disappear. I'm getting a different backtrace though and it doesn't seem to be related to ACL: Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libssl.so.8...done. Loaded symbols for /usr/local/lib/libssl.so.8 Reading symbols from /usr/local/lib/libcrypto.so.8...done. Loaded symbols for /usr/local/lib/libcrypto.so.8 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /lib/libthr.so.3...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/lib/libz.so...done. Loaded symbols for /usr/lib/libz.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 143 return (_c < 0 || _c >= __mb_sb_limit) ? _c : [New Thread 10e041c0 (LWP 103332/initial thread)] (gdb) bt full #0 0x00000000104b99a6 in hash_table_destroy (_table=0x28) at _ctype.h:143 table = (struct hash_table *) 0x10e29700 #1 0x0000000010478cf1 in settings_parser_deinit (_ctx=0x10e29778) at array.h:197 ctx = (struct setting_parser_context *) 0x0 #2 0x00000000104ac4ea in master_service_settings_cache_deinit (_cache=0x1045da08) at master-service-settings-cache.c:81 cache = (struct master_service_settings_cache *) 0x10e2e040 entry = (struct settings_entry *) 0x10e29740 next = (struct settings_entry *) 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x000000001044d1df in login_settings_deinit () at login-settings.c:129 No locals. #4 0x000000001044dd7e in main_deinit () at main.c:62 No locals. #5 0x000000001044df5c in login_binary_run (binary=0x4051c0, argc=2, argv=0x7fffffffebd8) at main.c:62 service_flags = 72 set_pool = 0x10e2d020 allow_core_dumps = true login_socket = 0x10457491 "login" c = -1 #6 0x00000000004036ee in main (argc=2, argv=0x7fffffffebd8) at client.c:40 No locals. Cheers, Olivier From stephan at rename-it.nl Wed Sep 19 21:07:12 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 19 Sep 2012 20:07:12 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <5059F145.1030803@fi.upm.es> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> Message-ID: <505A09D0.6040607@rename-it.nl> On 9/19/2012 6:22 PM, Juan C. Blanco wrote: > On 18/09/2012 20:49, Stephan Bosch wrote: >> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>> It seems that the error is related to the ld.gold fix, If I revert the >>> c52a0c561311 patch the error is not produced >> >> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >> but I forgot the Dovecot v2.1 tree. Fixed now: >> >> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c D'oh, this is the v2.2 change. The URL I meant is this one: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >> >> This only happens when you compile against installed Dovecot headers, >> people using --with-dovecot= (like me) would not have seen this. >> >> I'll push out a new release once 2.1.10 comes out. I'm being told that >> is imminent. > > Sorry, but using the new 0.3.3 version I'm getting the same error and > I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in > configure, not the installed headers. If needed I can send you the > configure invocation command. Yes, that could be helpful. Regards, Stephan. From florob at babelmonkeys.de Wed Sep 19 21:42:50 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Wed, 19 Sep 2012 20:42:50 +0200 Subject: [Dovecot] [PATCH] Generalize HMAC implementation In-Reply-To: <505A042D.5090705@babelmonkeys.de> References: <504647B2.5060804@babelmonkeys.de> <15E1301B-8902-4542-9562-DBAA5F825153@iki.fi> <504F7DC9.3040408@babelmonkeys.de> <147E1431-7B2B-427E-8929-0B83A3D3DDAA@dovecot.fi> <505A042D.5090705@babelmonkeys.de> Message-ID: <505A122A.3070307@babelmonkeys.de> Am 19.09.2012 19:43, schrieb Florian Zeitz: > Am 11.09.2012 20:21, schrieb Timo Sirainen: >> On 11.9.2012, at 21.07, Florian Zeitz wrote: >> >>>> I think we could simply #define the largest allowed context_size, use it for these buffers' sizes and then add i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE) >>>> >>> Well, either that, or we could use a union of all known context structs >>> there. >> >> That annoyingly requires #including in hmac.h all the different .h files that contain hash contexts. When new hashes are added I doubt anyone remembers to update that. And since the hmac_context is rather rarely used I don't think it matters if we just use a somewhat large context size. >> >>> Or we could use i_malloc() and i_free() under the assumption hmac_init() >>> + hmac_final() calls are always matched. >> >> Yeah, that's not too bad either I guess. Although it is different from other hash_init()+final calls which don't require it, and it's also wasting CPU a bit. >> > Here is an updated version of the patch. > It uses the variant defining the largest allowed content size. > I choose 1024 bit (128 Byte), since that is what most of the SHA-3 > candidates use, so it's hopefully somewhat future proof. Forget I said that. Unfortunately the MD5 context is already bigger (the SHA-1 one isn't). So 256 Byte it is I guess. Sorry for the lack of thorough testing. I hope I didn't miss another hash function. -------------- next part -------------- # HG changeset patch # User Florian Zeitz # Date 1346280236 -7200 # Node ID 5e6802858a5bb3a9b278f1cbdabe5c0ffb15248d # Parent 488c7dde19ad8baaa1d6bb48ee092c24eb9d44fb lib: Generalize hmac to be hash independent diff --git a/src/auth/auth-token.c b/src/auth/auth-token.c --- a/src/auth/auth-token.c +++ b/src/auth/auth-token.c @@ -11,7 +11,8 @@ #include "auth-common.h" #include "hex-binary.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "read-full.h" #include "write-full.h" @@ -168,16 +169,17 @@ const char *auth_token_get(const char *service, const char *session_pid, const char *username, const char *session_id) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char result[SHA1_RESULTLEN]; - hmac_sha1_init(&ctx, username, strlen(username)); - hmac_sha1_update(&ctx, session_pid, strlen(session_pid)); + hmac_init(&ctx, (const unsigned char*)username, strlen(username), + &hash_method_sha1); + hmac_update(&ctx, session_pid, strlen(session_pid)); if (session_id != NULL && *session_id != '\0') - hmac_sha1_update(&ctx, session_id, strlen(session_id)); - hmac_sha1_update(&ctx, service, strlen(service)); - hmac_sha1_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); - hmac_sha1_final(&ctx, result); + hmac_update(&ctx, session_id, strlen(session_id)); + hmac_update(&ctx, service, strlen(service)); + hmac_update(&ctx, auth_token_secret, sizeof(auth_token_secret)); + hmac_final(&ctx, result); return binary_to_hex(result, sizeof(result)); } diff --git a/src/auth/mech-cram-md5.c b/src/auth/mech-cram-md5.c --- a/src/auth/mech-cram-md5.c +++ b/src/auth/mech-cram-md5.c @@ -7,7 +7,9 @@ #include "ioloop.h" #include "buffer.h" #include "hex-binary.h" -#include "hmac-md5.h" +#include "hmac-cram-md5.h" +#include "hmac.h" +#include "md5.h" #include "randgen.h" #include "mech.h" #include "passdb.h" @@ -50,7 +52,7 @@ { unsigned char digest[MD5_RESULTLEN]; - struct hmac_md5_context ctx; + struct hmac_context ctx; const char *response_hex; if (size != CRAM_MD5_CONTEXTLEN) { @@ -59,9 +61,10 @@ return FALSE; } + hmac_init(&ctx, NULL, 0, &hash_method_md5); hmac_md5_set_cram_context(&ctx, credentials); - hmac_md5_update(&ctx, request->challenge, strlen(request->challenge)); - hmac_md5_final(&ctx, digest); + hmac_update(&ctx, request->challenge, strlen(request->challenge)); + hmac_final(&ctx, digest); response_hex = binary_to_hex(digest, sizeof(digest)); diff --git a/src/auth/mech-scram-sha1.c b/src/auth/mech-scram-sha1.c --- a/src/auth/mech-scram-sha1.c +++ b/src/auth/mech-scram-sha1.c @@ -9,7 +9,8 @@ #include "auth-common.h" #include "base64.h" #include "buffer.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "str.h" @@ -44,23 +45,23 @@ const unsigned char *salt, size_t salt_size, unsigned int i, unsigned char result[SHA1_RESULTLEN]) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; unsigned char U[SHA1_RESULTLEN]; unsigned int j, k; /* Calculate U1 */ - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, salt, salt_size); - hmac_sha1_update(&ctx, "\0\0\0\1", 4); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, salt, salt_size); + hmac_update(&ctx, "\0\0\0\1", 4); + hmac_final(&ctx, U); memcpy(result, U, SHA1_RESULTLEN); /* Calculate U2 to Ui and Hi */ for (j = 2; j <= i; j++) { - hmac_sha1_init(&ctx, str, str_size); - hmac_sha1_update(&ctx, U, sizeof(U)); - hmac_sha1_final(&ctx, U); + hmac_init(&ctx, str, str_size, &hash_method_sha1); + hmac_update(&ctx, U, sizeof(U)); + hmac_final(&ctx, U); for (k = 0; k < SHA1_RESULTLEN; k++) result[k] ^= U[k]; } @@ -94,7 +95,7 @@ static const char *get_scram_server_final(struct scram_auth_request *request) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char server_key[SHA1_RESULTLEN]; unsigned char server_signature[SHA1_RESULTLEN]; @@ -104,17 +105,17 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Server Key", 10); - hmac_sha1_final(&ctx, server_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Server Key", 10); + hmac_final(&ctx, server_key); safe_memset(request->salted_password, 0, sizeof(request->salted_password)); - hmac_sha1_init(&ctx, server_key, sizeof(server_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, server_signature); + hmac_init(&ctx, server_key, sizeof(server_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, server_signature); str = t_str_new(MAX_BASE64_ENCODED_SIZE(sizeof(server_signature))); str_append(str, "v="); @@ -213,7 +214,7 @@ static bool verify_credentials(struct scram_auth_request *request, const unsigned char *credentials, size_t size) { - struct hmac_sha1_context ctx; + struct hmac_context ctx; const char *auth_message; unsigned char client_key[SHA1_RESULTLEN]; unsigned char client_signature[SHA1_RESULTLEN]; @@ -224,10 +225,10 @@ Hi(credentials, size, request->salt, sizeof(request->salt), SCRAM_ITERATE_COUNT, request->salted_password); - hmac_sha1_init(&ctx, request->salted_password, - sizeof(request->salted_password)); - hmac_sha1_update(&ctx, "Client Key", 10); - hmac_sha1_final(&ctx, client_key); + hmac_init(&ctx, request->salted_password, + sizeof(request->salted_password), &hash_method_sha1); + hmac_update(&ctx, "Client Key", 10); + hmac_final(&ctx, client_key); sha1_get_digest(client_key, sizeof(client_key), stored_key); @@ -235,9 +236,9 @@ request->server_first_message, ",", request->client_final_message_without_proof, NULL); - hmac_sha1_init(&ctx, stored_key, sizeof(stored_key)); - hmac_sha1_update(&ctx, auth_message, strlen(auth_message)); - hmac_sha1_final(&ctx, client_signature); + hmac_init(&ctx, stored_key, sizeof(stored_key), &hash_method_sha1); + hmac_update(&ctx, auth_message, strlen(auth_message)); + hmac_final(&ctx, client_signature); for (i = 0; i < sizeof(client_signature); i++) client_signature[i] ^= client_key[i]; diff --git a/src/auth/password-scheme.c b/src/auth/password-scheme.c --- a/src/auth/password-scheme.c +++ b/src/auth/password-scheme.c @@ -6,7 +6,8 @@ #include "hex-binary.h" #include "md4.h" #include "md5.h" -#include "hmac-md5.h" +#include "hmac.h" +#include "hmac-cram-md5.h" #include "ntlm.h" #include "mycrypt.h" #include "randgen.h" @@ -655,12 +656,12 @@ cram_md5_generate(const char *plaintext, const char *user ATTR_UNUSED, const unsigned char **raw_password_r, size_t *size_r) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char *context_digest; context_digest = t_malloc(CRAM_MD5_CONTEXTLEN); - hmac_md5_init(&ctx, (const unsigned char *)plaintext, - strlen(plaintext)); + hmac_init(&ctx, (const unsigned char *)plaintext, + strlen(plaintext), &hash_method_md5); hmac_md5_get_cram_context(&ctx, context_digest); *raw_password_r = context_digest; diff --git a/src/lib-imap-urlauth/imap-urlauth.c b/src/lib-imap-urlauth/imap-urlauth.c --- a/src/lib-imap-urlauth/imap-urlauth.c +++ b/src/lib-imap-urlauth/imap-urlauth.c @@ -3,7 +3,8 @@ #include "lib.h" #include "hostpid.h" #include "var-expand.h" -#include "hmac-sha1.h" +#include "hmac.h" +#include "sha1.h" #include "randgen.h" #include "safe-memset.h" #include "mail-storage.h" @@ -88,15 +89,15 @@ const unsigned char mailbox_key[IMAP_URLAUTH_KEY_LEN], size_t *token_len_r) { - struct hmac_sha1_context hmac; + struct hmac_context hmac; unsigned char *token; token = t_new(unsigned char, SHA1_RESULTLEN + 1); token[0] = IMAP_URLAUTH_MECH_INTERNAL_VERSION; - hmac_sha1_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN); - hmac_sha1_update(&hmac, rumpurl, strlen(rumpurl)); - hmac_sha1_final(&hmac, token+1); + hmac_init(&hmac, mailbox_key, IMAP_URLAUTH_KEY_LEN, &hash_method_sha1); + hmac_update(&hmac, rumpurl, strlen(rumpurl)); + hmac_final(&hmac, token+1); *token_len_r = SHA1_RESULTLEN + 1; return token; diff --git a/src/lib-ntlm/ntlm-encrypt.c b/src/lib-ntlm/ntlm-encrypt.c --- a/src/lib-ntlm/ntlm-encrypt.c +++ b/src/lib-ntlm/ntlm-encrypt.c @@ -11,7 +11,8 @@ #include "compat.h" #include "safe-memset.h" #include "md4.h" -#include "hmac-md5.h" +#include "md5.h" +#include "hmac.h" #include "ntlm.h" #include "ntlm-des.h" @@ -60,12 +61,12 @@ } static void -hmac_md5_ucs2le_string_ucase(struct hmac_md5_context *ctx, const char *str) +hmac_md5_ucs2le_string_ucase(struct hmac_context *ctx, const char *str) { size_t len; unsigned char *wstr = t_unicode_str(str, 1, &len); - hmac_md5_update(ctx, wstr, len); + hmac_update(ctx, wstr, len); } static void ATTR_NULL(2) @@ -73,13 +74,13 @@ const unsigned char *hash_v1, unsigned char hash[NTLMSSP_V2_HASH_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; - hmac_md5_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE); + hmac_init(&ctx, hash_v1, NTLMSSP_HASH_SIZE, &hash_method_md5); hmac_md5_ucs2le_string_ucase(&ctx, user); if (target != NULL) hmac_md5_ucs2le_string_ucase(&ctx, target); - hmac_md5_final(&ctx, hash); + hmac_final(&ctx, hash); } void @@ -124,15 +125,15 @@ const unsigned char *blob, size_t blob_size, unsigned char response[NTLMSSP_V2_RESPONSE_SIZE]) { - struct hmac_md5_context ctx; + struct hmac_context ctx; unsigned char hash[NTLMSSP_V2_HASH_SIZE]; ntlm_v2_hash(user, target, hash_v1, hash); - hmac_md5_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE); - hmac_md5_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); - hmac_md5_update(&ctx, blob, blob_size); - hmac_md5_final(&ctx, response); + hmac_init(&ctx, hash, NTLMSSP_V2_HASH_SIZE, &hash_method_md5); + hmac_update(&ctx, challenge, NTLMSSP_CHALLENGE_SIZE); + hmac_update(&ctx, blob, blob_size); + hmac_final(&ctx, response); safe_memset(hash, 0, sizeof(hash)); } diff --git a/src/lib/Makefile.am b/src/lib/Makefile.am --- a/src/lib/Makefile.am +++ b/src/lib/Makefile.am @@ -44,8 +44,8 @@ hash2.c \ hex-binary.c \ hex-dec.c \ - hmac-md5.c \ - hmac-sha1.c \ + hmac.c \ + hmac-cram-md5.c \ home-expand.c \ hostpid.c \ imem.c \ @@ -170,8 +170,8 @@ hash2.h \ hex-binary.h \ hex-dec.h \ - hmac-md5.h \ - hmac-sha1.h \ + hmac.h \ + hmac-cram-md5.h \ home-expand.h \ hostpid.h \ imem.h \ diff --git a/src/lib/hmac-cram-md5.c b/src/lib/hmac-cram-md5.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.c @@ -0,0 +1,63 @@ +/* + * CRAM-MD5 (RFC 2195) compatibility code + * Copyright (c) 2003 Joshua Goodall + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "md5.h" +#include "hmac-cram-md5.h" + +void hmac_md5_get_cram_context(struct hmac_context *hmac_ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDPUT(p, c) STMT_START { \ + *(p)++ = (c) & 0xff; \ + *(p)++ = (c) >> 8 & 0xff; \ + *(p)++ = (c) >> 16 & 0xff; \ + *(p)++ = (c) >> 24 & 0xff; \ +} STMT_END + cdp = context_digest; + CDPUT(cdp, ctxo->a); + CDPUT(cdp, ctxo->b); + CDPUT(cdp, ctxo->c); + CDPUT(cdp, ctxo->d); + CDPUT(cdp, ctx->a); + CDPUT(cdp, ctx->b); + CDPUT(cdp, ctx->c); + CDPUT(cdp, ctx->d); +} + +void hmac_md5_set_cram_context(struct hmac_context *hmac_ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) +{ + const unsigned char *cdp; + + struct md5_context *ctx = (void*)hmac_ctx->ctx; + struct md5_context *ctxo = (void*)hmac_ctx->ctxo; + +#define CDGET(p, c) STMT_START { \ + (c) = (*p++); \ + (c) += (*p++ << 8); \ + (c) += (*p++ << 16); \ + (c) += (*p++ << 24); \ +} STMT_END + cdp = context_digest; + CDGET(cdp, ctxo->a); + CDGET(cdp, ctxo->b); + CDGET(cdp, ctxo->c); + CDGET(cdp, ctxo->d); + CDGET(cdp, ctx->a); + CDGET(cdp, ctx->b); + CDGET(cdp, ctx->c); + CDGET(cdp, ctx->d); + + ctxo->lo = ctx->lo = 64; + ctxo->hi = ctx->hi = 0; +} diff --git a/src/lib/hmac-cram-md5.h b/src/lib/hmac-cram-md5.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac-cram-md5.h @@ -0,0 +1,14 @@ +#ifndef HMAC_CRAM_MD5_H +#define HMAC_CRAM_MD5_H + +#include "hmac.h" + +#define CRAM_MD5_CONTEXTLEN 32 + +void hmac_md5_get_cram_context(struct hmac_context *ctx, + unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); +void hmac_md5_set_cram_context(struct hmac_context *ctx, + const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); + + +#endif diff --git a/src/lib/hmac-md5.c b/src/lib/hmac-md5.c deleted file mode 100644 --- a/src/lib/hmac-md5.c +++ /dev/null @@ -1,101 +0,0 @@ -/* - * HMAC-MD5 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * - * CRAM-MD5 (RFC 2195) compatibility code - * Copyright (c) 2003 Joshua Goodall - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-md5.h" -#include "safe-memset.h" - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len) -{ - int i; - unsigned char md5key[16]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - md5_get_digest(key, key_len, md5key); - key = md5key; - key_len = 16; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - md5_init(&ctx->ctx); - md5_update(&ctx->ctx, k_ipad, 64); - md5_init(&ctx->ctxo); - md5_update(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_md5_final(struct hmac_md5_context *ctx, unsigned char *digest) -{ - md5_final(&ctx->ctx, digest); - - md5_update(&ctx->ctxo, digest, 16); - md5_final(&ctx->ctxo, digest); -} - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - unsigned char *cdp; - -#define CDPUT(p, c) STMT_START { \ - *(p)++ = (c) & 0xff; \ - *(p)++ = (c) >> 8 & 0xff; \ - *(p)++ = (c) >> 16 & 0xff; \ - *(p)++ = (c) >> 24 & 0xff; \ -} STMT_END - cdp = context_digest; - CDPUT(cdp, ctx->ctxo.a); - CDPUT(cdp, ctx->ctxo.b); - CDPUT(cdp, ctx->ctxo.c); - CDPUT(cdp, ctx->ctxo.d); - CDPUT(cdp, ctx->ctx.a); - CDPUT(cdp, ctx->ctx.b); - CDPUT(cdp, ctx->ctx.c); - CDPUT(cdp, ctx->ctx.d); -} - -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]) -{ - const unsigned char *cdp; - -#define CDGET(p, c) STMT_START { \ - (c) = (*p++); \ - (c) += (*p++ << 8); \ - (c) += (*p++ << 16); \ - (c) += (*p++ << 24); \ -} STMT_END - cdp = context_digest; - CDGET(cdp, ctx->ctxo.a); - CDGET(cdp, ctx->ctxo.b); - CDGET(cdp, ctx->ctxo.c); - CDGET(cdp, ctx->ctxo.d); - CDGET(cdp, ctx->ctx.a); - CDGET(cdp, ctx->ctx.b); - CDGET(cdp, ctx->ctx.c); - CDGET(cdp, ctx->ctx.d); - - ctx->ctxo.lo = ctx->ctx.lo = 64; - ctx->ctxo.hi = ctx->ctx.hi = 0; -} diff --git a/src/lib/hmac-md5.h b/src/lib/hmac-md5.h deleted file mode 100644 --- a/src/lib/hmac-md5.h +++ /dev/null @@ -1,29 +0,0 @@ -#ifndef HMAC_MD5_H -#define HMAC_MD5_H - -#include "md5.h" - -#define CRAM_MD5_CONTEXTLEN 32 - -struct hmac_md5_context { - struct md5_context ctx, ctxo; -}; - -void hmac_md5_init(struct hmac_md5_context *ctx, - const unsigned char *key, size_t key_len); -void hmac_md5_final(struct hmac_md5_context *ctx, - unsigned char digest[MD5_RESULTLEN]); - -void hmac_md5_get_cram_context(struct hmac_md5_context *ctx, - unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); -void hmac_md5_set_cram_context(struct hmac_md5_context *ctx, - const unsigned char context_digest[CRAM_MD5_CONTEXTLEN]); - - -static inline void -hmac_md5_update(struct hmac_md5_context *ctx, const void *data, size_t size) -{ - md5_update(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac-sha1.c b/src/lib/hmac-sha1.c deleted file mode 100644 --- a/src/lib/hmac-sha1.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * HMAC-SHA1 (RFC-2104) implementation. - * - * Copyright (c) 2004 Andrey Panin - * Copyright (c) 2011 Florian Zeitz - * - * This software is released under the MIT license. - */ - -#include "lib.h" -#include "hmac-sha1.h" -#include "safe-memset.h" - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len) -{ - int i; - unsigned char sha1key[20]; - unsigned char k_ipad[64]; - unsigned char k_opad[64]; - - if (key_len > 64) { - sha1_get_digest(key, key_len, sha1key); - key = sha1key; - key_len = 20; - } - - memcpy(k_ipad, key, key_len); - memset(k_ipad + key_len, 0, 64 - key_len); - memcpy(k_opad, k_ipad, 64); - - for (i = 0; i < 64; i++) { - k_ipad[i] ^= 0x36; - k_opad[i] ^= 0x5c; - } - - sha1_init(&ctx->ctx); - sha1_loop(&ctx->ctx, k_ipad, 64); - sha1_init(&ctx->ctxo); - sha1_loop(&ctx->ctxo, k_opad, 64); - - safe_memset(k_ipad, 0, 64); - safe_memset(k_opad, 0, 64); -} - -void hmac_sha1_final(struct hmac_sha1_context *ctx, unsigned char *digest) -{ - sha1_result(&ctx->ctx, digest); - - sha1_loop(&ctx->ctxo, digest, 20); - sha1_result(&ctx->ctxo, digest); -} diff --git a/src/lib/hmac-sha1.h b/src/lib/hmac-sha1.h deleted file mode 100644 --- a/src/lib/hmac-sha1.h +++ /dev/null @@ -1,22 +0,0 @@ -#ifndef HMAC_SHA1_H -#define HMAC_SHA1_H - -#include "sha1.h" - -struct hmac_sha1_context { - struct sha1_ctxt ctx, ctxo; -}; - -void hmac_sha1_init(struct hmac_sha1_context *ctx, - const void *key, size_t key_len); -void hmac_sha1_final(struct hmac_sha1_context *ctx, - unsigned char digest[SHA1_RESULTLEN]); - - -static inline void -hmac_sha1_update(struct hmac_sha1_context *ctx, const void *data, size_t size) -{ - sha1_loop(&ctx->ctx, data, size); -} - -#endif diff --git a/src/lib/hmac.c b/src/lib/hmac.c new file mode 100644 --- /dev/null +++ b/src/lib/hmac.c @@ -0,0 +1,58 @@ +/* + * HMAC (RFC-2104) implementation. + * + * Copyright (c) 2004 Andrey Panin + * Copyright (c) 2011-2012 Florian Zeitz + * + * This software is released under the MIT license. + */ + +#include "lib.h" +#include "hmac.h" +#include "safe-memset.h" + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth) +{ + int i; + unsigned char k_ipad[64]; + unsigned char k_opad[64]; + unsigned char hashedkey[meth->digest_size]; + + i_assert(meth->context_size <= HMAC_MAX_CONTEXT_SIZE); + + ctx->hash = meth; + + if (key_len > 64) { + meth->init(ctx->ctx); + meth->loop(ctx->ctx, key, key_len); + meth->result(ctx->ctx, hashedkey); + key = hashedkey; + key_len = meth->digest_size; + } + + memcpy(k_ipad, key, key_len); + memset(k_ipad + key_len, 0, 64 - key_len); + memcpy(k_opad, k_ipad, 64); + + for (i = 0; i < 64; i++) { + k_ipad[i] ^= 0x36; + k_opad[i] ^= 0x5c; + } + + meth->init(ctx->ctx); + meth->loop(ctx->ctx, k_ipad, 64); + meth->init(ctx->ctxo); + meth->loop(ctx->ctxo, k_opad, 64); + + safe_memset(k_ipad, 0, 64); + safe_memset(k_opad, 0, 64); +} + +void hmac_final(struct hmac_context *ctx, unsigned char *digest) +{ + ctx->hash->result(ctx->ctx, digest); + + ctx->hash->loop(ctx->ctxo, digest, ctx->hash->digest_size); + ctx->hash->result(ctx->ctxo, digest); +} diff --git a/src/lib/hmac.h b/src/lib/hmac.h new file mode 100644 --- /dev/null +++ b/src/lib/hmac.h @@ -0,0 +1,26 @@ +#ifndef HMAC_H +#define HMAC_H + +#include "hash-method.h" +#include "sha1.h" + +#define HMAC_MAX_CONTEXT_SIZE 256 + +struct hmac_context { + char ctx[HMAC_MAX_CONTEXT_SIZE]; + char ctxo[HMAC_MAX_CONTEXT_SIZE]; + const struct hash_method *hash; +}; + +void hmac_init(struct hmac_context *ctx, const unsigned char *key, + size_t key_len, const struct hash_method *meth); +void hmac_final(struct hmac_context *ctx, unsigned char *digest); + + +static inline void +hmac_update(struct hmac_context *ctx, const void *data, size_t size) +{ + ctx->hash->loop(ctx->ctx, data, size); +} + +#endif From c at roessner-network-solutions.com Wed Sep 19 22:48:17 2012 From: c at roessner-network-solutions.com (=?iso-8859-1?Q?Christian_R=F6=DFner?=) Date: Wed, 19 Sep 2012 21:48:17 +0200 Subject: [Dovecot] OT: z-Push In-Reply-To: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> References: <2527D796-B1BC-4D21-B59E-35DC9A7C46F5@roessner-network-solutions.com> Message-ID: Hi, > just a short question: > > I have installed and configured z-Push to use BackendIMAP. On my iPhone I have created an account and I can see my mails. Also sending works. So far so good. Now the question: > > When I create a new folder i.e. Test under the root (/), then I get an error. And after resyncing the folder has been created not under /, but under /0/Test. > > Also I can not remove that folder over ActiveSync. > > Does somebody have z-Push fully working with Dovecot or is z-Push designed for Cyrus and only some features are working with Dovecot? > > So at the moment folder create/move/delete does not work here with z-Push-2.0.3 and Dovecot 2.1.9 unfortunately z-Push is much too buggy here. So I dropped it in favor of standard IMAP. Best wishes -Christian R??ner --- Bachelor of Science Informatik Erlenwiese 14, 36304 Alsfeld T: +49 6631 78823400, F: +49 6631 78823409, M: +49 176 93118939 USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com From ben at indietorrent.org Thu Sep 20 00:41:38 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 17:41:38 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <504F4D5D.8040207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> Message-ID: <505A3C12.7050207@indietorrent.org> On 9/11/2012 10:40 AM, Ben Johnson wrote: > You nailed it, Ben. Thanks! > > I downloaded/installed the dovecot-dev package from the same maintainer > and was able to build Antispam without issue. > > The hang-up now seems to be with my configuration. > > I'm seeing the following error in my email client whenever I attempt to > move an email message from, say, Inbox to Junk folder: > > [SERVERBUG] BUG Unknown internal error. > > Nothing at all appears in syslog when this happens. > > I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam > , to the letter, I believe. > > The relevant portions of my configuration are as follows: > > --------------------- > protocol imap { > mail_plugins = quota imap_quota antispam > } > > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > > antispam_backend = MAILTRAIN > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > antispam_spam_pattern_ignorecase = spam;junk > } > --------------------- > > The permissions and execute bit are set in accordance with other > executables in the same directory: > > # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" > -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh > > I would provide more log content, but the debug options seem to be gone > in the forked, >= 2.0 version of this plug-in. > > Any ideas are very much appreciated. > > Thanks, > > -Ben Given that nobody shared any insight here, I decided to stand-up a completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. While the error is different this time around, I am still unable to get any variant of the dovecot-antispam plug-in working. When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and I use the above configuration, my syslog fills-up with the following: dovecot: imap-login: Login: user=, method=PLAIN, rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS dovecot: master: Error: service(imap): child 2879 returned error 3 Surprisingly, I can find only two other mentions of "error 3" anywhere on the Internet. Those mentions were from another user to this mailing list: http://www.dovecot.org/list/dovecot/2012-August/067787.html http://www.dovecot.org/list/dovecot/2012-August/067814.html The lack of "error 3" ubiquity leads me to believe that I must be doing something stupid. There's no question that this behavior is related to the antispam-plugin, because when I commented-out the relevant lines in Dovecot's configuration file, the error does not occur. I have followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the result. The only instance in which I diverged from the Wiki article is that I used the Ubuntu-provided dovecot-antispam package (instead of compiling the plug-in from source). I am trying to use the "mailtrain" back-end, per the above-cited Wiki article, to call SpamAssassin's "sa-learn" utility for real-time learning. I am not using the SpamAssassin daemon (spamd). I'm about to throw in the towel on this plug-in, so any help would be greatly appreciated. Thank you, -Ben From tom at whyscream.net Thu Sep 20 01:19:07 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 20 Sep 2012 00:19:07 +0200 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505A44DB.6020200@whyscream.net> On 19/09/12 23:41, Ben Johnson wrote: > > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben There are 2 versions of the dovecot-antispam package. The original version was created by Johannes Berg, then died almost a silent death, and was forked by Eugene Paskevitch. Later Johannes revived his original version somewhat. The ubuntu package is based on the 'revived Johannes fork' [1], while the documentation on the dovecot wiki describes the 'Eugene fork'. [2] [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright [2] http://wiki2.dovecot.org/Plugins/Antispam Thus you'll need to choose which plugin you want, then follow the complete procedure for that plugin only. They both can do what you want, but building and configuration are slightly different. It would be nice if Johannes and Eugene would either agree on a single version of the plugin, or rename one of the forks so they don't get mixed up that easily. I'm not sure if that has been discussed (on- or off-list) in the past. FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to upgrade to 2.1 yet, so no experience there. -- Tom From ben at indietorrent.org Thu Sep 20 02:17:43 2012 From: ben at indietorrent.org (Ben Johnson) Date: Wed, 19 Sep 2012 19:17:43 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A44DB.6020200@whyscream.net> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> Message-ID: <505A5297.8050808@indietorrent.org> On 9/19/2012 6:19 PM, Tom Hendrikx wrote: > On 19/09/12 23:41, Ben Johnson wrote: >> >> >> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>> You nailed it, Ben. Thanks! >>> >>> I downloaded/installed the dovecot-dev package from the same maintainer >>> and was able to build Antispam without issue. >>> >>> The hang-up now seems to be with my configuration. >>> >>> I'm seeing the following error in my email client whenever I attempt to >>> move an email message from, say, Inbox to Junk folder: >>> >>> [SERVERBUG] BUG Unknown internal error. >>> >>> Nothing at all appears in syslog when this happens. >>> >>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>> , to the letter, I believe. >>> >>> The relevant portions of my configuration are as follows: >>> >>> --------------------- >>> protocol imap { >>> mail_plugins = quota imap_quota antispam >>> } >>> >>> plugin { >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> sieve = /var/vmail/%d/%n/.sieve >>> >>> antispam_backend = MAILTRAIN >>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>> antispam_mail_spam = --spam >>> antispam_mail_notspam = --ham >>> antispam_spam_pattern_ignorecase = spam;junk >>> } >>> --------------------- >>> >>> The permissions and execute bit are set in accordance with other >>> executables in the same directory: >>> >>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>> >>> I would provide more log content, but the debug options seem to be gone >>> in the forked, >= 2.0 version of this plug-in. >>> >>> Any ideas are very much appreciated. >>> >>> Thanks, >>> >>> -Ben >> >> Given that nobody shared any insight here, I decided to stand-up a >> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >> >> While the error is different this time around, I am still unable to get >> any variant of the dovecot-antispam plug-in working. >> >> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >> I use the above configuration, my syslog fills-up with the following: >> >> dovecot: imap-login: Login: user=, method=PLAIN, >> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >> dovecot: master: Error: service(imap): child 2879 returned error 3 >> >> Surprisingly, I can find only two other mentions of "error 3" anywhere >> on the Internet. Those mentions were from another user to this mailing list: >> >> http://www.dovecot.org/list/dovecot/2012-August/067787.html >> http://www.dovecot.org/list/dovecot/2012-August/067814.html >> >> The lack of "error 3" ubiquity leads me to believe that I must be doing >> something stupid. >> >> There's no question that this behavior is related to the >> antispam-plugin, because when I commented-out the relevant lines in >> Dovecot's configuration file, the error does not occur. >> >> I have followed the instructions at >> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >> result. >> >> The only instance in which I diverged from the Wiki article is that I >> used the Ubuntu-provided dovecot-antispam package (instead of compiling >> the plug-in from source). >> >> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >> >> I am not using the SpamAssassin daemon (spamd). >> >> I'm about to throw in the towel on this plug-in, so any help would be >> greatly appreciated. >> >> Thank you, >> >> -Ben > > There are 2 versions of the dovecot-antispam package. The original > version was created by Johannes Berg, then died almost a silent death, > and was forked by Eugene Paskevitch. Later Johannes revived his original > version somewhat. > > The ubuntu package is based on the 'revived Johannes fork' [1], while > the documentation on the dovecot wiki describes the 'Eugene fork'. [2] > > [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright > [2] http://wiki2.dovecot.org/Plugins/Antispam > > Thus you'll need to choose which plugin you want, then follow the > complete procedure for that plugin only. They both can do what you want, > but building and configuration are slightly different. > > It would be nice if Johannes and Eugene would either agree on a single > version of the plugin, or rename one of the forks so they don't get > mixed up that easily. I'm not sure if that has been discussed (on- or > off-list) in the past. > > FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the > 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to > upgrade to 2.1 yet, so no experience there. > > -- > Tom > Thank you for your timely response, Tom. Much appreciated. Your remarks clear-up a lot of the confusion for me. I didn't realize that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did I realize that this is the version that is packaged with Ubuntu 12.04. Given that I'm more-or-less forced to work with Ubuntu LTS, I would prefer to use the OS's package management system wherever possible. I suppose, by extension, that I would prefer to use the "revived Johannes fork". I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but am unable to get either to work the way I want (mailtrain back-end with real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki article). Shouldn't the basic tenets of each version of the plug-in apply? Where should I start, in the way of seeking support? Basically, I intend to use the Johannes version, but I want to configure it in the manner described in the Eugene version Wiki article. I've already described (rather exhaustively) what happens when I attempt to use the Johannes version and adapt the Eugene version instructions. I'm curious to know what method you've used over the years with such glowing success. Which back-end did you use with Dovecot 1/Johannes? How about with Dovecot 2/Eugene? Thanks again, -Ben From selcuk.yazar at gmail.com Thu Sep 20 09:21:21 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:21:21 +0300 Subject: [Dovecot] Prevent Download messages from server Message-ID: Hi, can we prevent download messages from server user by user ? sme common used mail's message must be remain at the server, but sometimes we download them ? thanks in advance -- Sel?uk YAZAR From robert at schetterer.org Thu Sep 20 09:28:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:28:37 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AB795.3000106@schetterer.org> Am 20.09.2012 08:21, schrieb Selcuk Yazar: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > please clear , what you mean with 1 : with download messages 2 : user by user 3 : sme 4 : mail's message must be remain at the server 5 : but sometimes we download them show your dovecot conf and problem logs -- Best Regards MfG Robert Schetterer From selcuk.yazar at gmail.com Thu Sep 20 09:31:19 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:31:19 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AB795.3000106@schetterer.org> References: <505AB795.3000106@schetterer.org> Message-ID: Hi, we have no problem, just i want to learn how can i do that. i think it's clear . sel?uk On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > Am 20.09.2012 08:21, schrieb Selcuk Yazar: > > Hi, > > > > can we prevent download messages from server user by user ? sme common > used > > mail's message must be remain at the server, but sometimes we download > them > > ? > > > > thanks in advance > > > > please clear , what you mean with > > 1 : with download messages > 2 : user by user > 3 : sme > 4 : mail's message must be remain at the server > 5 : but sometimes we download them > > show your dovecot conf and problem logs > > -- > Best Regards > MfG Robert Schetterer > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From robert at schetterer.org Thu Sep 20 09:38:48 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 08:38:48 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AB795.3000106@schetterer.org> Message-ID: <505AB9F8.6060603@schetterer.org> Am 20.09.2012 08:31, schrieb Selcuk Yazar: > Hi, > > we have no problem, just i want to learn how can i do that. i think it's > clear . > > > sel?uk ok ,so the right answer is, unless you give more details stop dovecot , nobody will able to connect to it > > On Thu, Sep 20, 2012 at 9:28 AM, Robert Schetterer wrote: > >> Am 20.09.2012 08:21, schrieb Selcuk Yazar: >>> Hi, >>> >>> can we prevent download messages from server user by user ? sme common >> used >>> mail's message must be remain at the server, but sometimes we download >> them >>> ? >>> >>> thanks in advance >>> >> >> please clear , what you mean with >> >> 1 : with download messages >> 2 : user by user >> 3 : sme >> 4 : mail's message must be remain at the server >> 5 : but sometimes we download them >> >> show your dovecot conf and problem logs >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > -- Best Regards MfG Robert Schetterer From alessio at skye.it Thu Sep 20 09:45:03 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:45:03 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ABB6F.2010606@skye.it> Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > If your users can log and read messages can obviously download it. If you use IMAP you can set ACL and enable read only for the folders: http://wiki2.dovecot.org/ACL -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From alessio at skye.it Thu Sep 20 09:51:14 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 08:51:14 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <505A0654.3010501@gmail.com> References: <505A0654.3010501@gmail.com> Message-ID: <505ABCE2.30802@skye.it> Il 19/09/2012 19:52, interfaSys s?rl ha scritto: > I see the same crashes on FreeBSD. Turning off ACL makes the crashes > disappear. > > I'm getting a different backtrace though and it doesn't seem to be > related to ACL: > > Core was generated by `imap-login'. > Thanks for your report. My backtrace was genereted from "dovecot-lda", but in both cases disable ACL "solve" the problem. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From selcuk.yazar at gmail.com Thu Sep 20 09:53:28 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 09:53:28 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ABB6F.2010606@skye.it> References: <505ABB6F.2010606@skye.it> Message-ID: Thank you, i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think it's diffucult to evaluate in POP3. also we are using openLDAP backend, if i change folder permissions , does it work ? sel?uk On Thu, Sep 20, 2012 at 9:45 AM, Alessio Cecchi wrote: > Il 20/09/2012 08:21, Selcuk Yazar ha scritto: > >> Hi, >> >> can we prevent download messages from server user by user ? sme common >> used >> mail's message must be remain at the server, but sometimes we download >> them >> ? >> >> thanks in advance >> >> > If your users can log and read messages can obviously download it. > > If you use IMAP you can set ACL and enable read only for the folders: > > http://wiki2.dovecot.org/ACL > > -- > Alessio Cecchi is: > @ ILS -> http://www.linux.it/~alessice/ > on LinkedIn -> http://www.linkedin.com/in/**alessice > Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ > @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From odhiambo at gmail.com Thu Sep 20 10:13:08 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Thu, 20 Sep 2012 10:13:08 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: On Thu, Sep 20, 2012 at 9:53 AM, Selcuk Yazar wrote: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Changing folder permissions to what? I suppose you understand what you mean by changing folder permissions! If you make them read-only (no write) for the users group, as that is what you could be thinking, you'll probably also affect the ability of the MTA to write to them (deliver mail). Please explain your situation holistically so that we can understand and give suggestions... -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. From alessio at skye.it Thu Sep 20 10:19:31 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 20 Sep 2012 09:19:31 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505ABB6F.2010606@skye.it> Message-ID: <505AC383.2010107@skye.it> Il 20/09/2012 08:53, Selcuk Yazar ha scritto: > Thank you, > > i looked up http://wiki2.dovecot.org/ACL section. We use POP3 , i think > it's diffucult to evaluate in POP3. > > also we are using openLDAP backend, if i change folder permissions , does > it work ? > > Change folder permissions is a bad idea, for example if you use Maildir/ as mailbox format you can't remove +w, messages should be moved from new/ to cur/, add flags to name and so on. I do not think it's possible to do what you ask. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From apm at one.com Thu Sep 20 10:24:34 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 20 Sep 2012 09:24:34 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505AC4B2.2040900@one.com> > we have no problem, just i want to learn how can i do that. i think > it's clear . Well... I'm pretty sure most others don't. But anyway. As in ALL Internet protocols (IMAP being no exception), letting the client read data on the server requires it to download the data. Preventing download will prevent reading the mail. Period. So if you're fine with that and just want to learn how to do it, then just disable the account in the user database. /Peter From selcuk.yazar at gmail.com Thu Sep 20 10:28:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:28:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: Alessio and Peter thank you again. maybe it can be future request. :) selcuk. On Thu, Sep 20, 2012 at 10:24 AM, Peter Mogensen wrote: > > we have no problem, just i want to learn how can i do that. i think > > it's clear . > > Well... I'm pretty sure most others don't. > > But anyway. As in ALL Internet protocols (IMAP being no exception), > letting the client read data on the server requires it to download the data. > Preventing download will prevent reading the mail. Period. > > So if you're fine with that and just want to learn how to do it, then just > disable the account in the user database. > > /Peter > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From jc at info-systems.de Thu Sep 20 10:51:41 2012 From: jc at info-systems.de (Jakob Curdes) Date: Thu, 20 Sep 2012 09:51:41 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: <505AC4B2.2040900@one.com> Message-ID: <505ACB0D.7020600@info-systems.de> Am 20.09.2012 09:28, schrieb Selcuk Yazar: > Alessio and Peter thank you again. maybe it can be future request. :) We are trying to tell you that it is impossible to write a mail _client_ that does not download (i.e. transfer) the mail that it shows to the user. This is why it makes no sense to prevent the download on the server side, and so there will never be such a function. Preventing the download (the transfer) of the mail fro mthe server to the client just means that the user *cannot read it*. Another question is whether there are mail _clients_ which do not offer a possibility to *save* the read mail (which has already been "downloaded" for reading it). But that question is not really related to the dovecot mail server. If you want to get users to read mails, but not download them, I would suggest you deny the direct access to the mail server and use a browser-based application that shows the entire mail to the user, maybe as a graphical representation. Or you use some sort of terminal software that connects to a host where the mail _clients_ itself are installed, just letting the user view the data but nor copy it, e.g. the VNC protocol. Hope this helps, Jakob Curdes From selcuk.yazar at gmail.com Thu Sep 20 10:54:20 2012 From: selcuk.yazar at gmail.com (Selcuk Yazar) Date: Thu, 20 Sep 2012 10:54:20 +0300 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505ACB0D.7020600@info-systems.de> References: <505AC4B2.2040900@one.com> <505ACB0D.7020600@info-systems.de> Message-ID: why are you trying ??? this is brain storming On Thu, Sep 20, 2012 at 10:51 AM, Jakob Curdes wrote: > Am 20.09.2012 09:28, schrieb Selcuk Yazar: > > Alessio and Peter thank you again. maybe it can be future request. :) > > We are trying to tell you that it is impossible to write a mail *client*that does not download (i.e. transfer) the mail that it shows to the user. > This is why it makes no sense to prevent the download on the server side, > and so there will never be such a function. > Preventing the download (the transfer) of the mail fro mthe server to the > client just means that the user *cannot read it*. > > Another question is whether there are mail *clients* which do not offer a > possibility to *save* the read mail (which has already been "downloaded" > for reading it). But that question is not really related to the dovecot > mail server. > > If you want to get users to read mails, but not download them, I would > suggest you deny the direct access to the mail server and use a > browser-based application that shows the entire mail to the user, maybe as > a graphical representation. Or you use some sort of terminal software > that connects to a host where the mail *clients* itself are installed, > just letting the user view the data but nor copy it, e.g. the VNC protocol. > > Hope this helps, > Jakob Curdes > > > -- Sel?uk YAZAR http://www.selcukyazar.blogspot.com From Bill at knoxvillechristian.org Thu Sep 20 11:32:54 2012 From: Bill at knoxvillechristian.org (Bill Shirley) Date: Thu, 20 Sep 2012 04:32:54 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A3C12.7050207@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> Message-ID: <505AD4B6.1020109@knoxvillechristian.org> On 9/19/2012 5:41 PM, Ben Johnson wrote: > > On 9/11/2012 10:40 AM, Ben Johnson wrote: >> You nailed it, Ben. Thanks! >> >> I downloaded/installed the dovecot-dev package from the same maintainer >> and was able to build Antispam without issue. >> >> The hang-up now seems to be with my configuration. >> >> I'm seeing the following error in my email client whenever I attempt to >> move an email message from, say, Inbox to Junk folder: >> >> [SERVERBUG] BUG Unknown internal error. >> >> Nothing at all appears in syslog when this happens. >> >> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >> , to the letter, I believe. >> >> The relevant portions of my configuration are as follows: >> >> --------------------- >> protocol imap { >> mail_plugins = quota imap_quota antispam >> } >> >> plugin { >> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> sieve = /var/vmail/%d/%n/.sieve >> >> antispam_backend = MAILTRAIN >> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >> antispam_mail_spam = --spam >> antispam_mail_notspam = --ham >> antispam_spam_pattern_ignorecase = spam;junk >> } >> --------------------- >> >> The permissions and execute bit are set in accordance with other >> executables in the same directory: >> >> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >> >> I would provide more log content, but the debug options seem to be gone >> in the forked, >= 2.0 version of this plug-in. >> >> Any ideas are very much appreciated. >> >> Thanks, >> >> -Ben > Given that nobody shared any insight here, I decided to stand-up a > completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. > > While the error is different this time around, I am still unable to get > any variant of the dovecot-antispam plug-in working. > > When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and > I use the above configuration, my syslog fills-up with the following: > > dovecot: imap-login: Login: user=, method=PLAIN, > rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS > dovecot: master: Error: service(imap): child 2879 returned error 3 > > Surprisingly, I can find only two other mentions of "error 3" anywhere > on the Internet. Those mentions were from another user to this mailing list: > > http://www.dovecot.org/list/dovecot/2012-August/067787.html > http://www.dovecot.org/list/dovecot/2012-August/067814.html > > The lack of "error 3" ubiquity leads me to believe that I must be doing > something stupid. > > There's no question that this behavior is related to the > antispam-plugin, because when I commented-out the relevant lines in > Dovecot's configuration file, the error does not occur. > > I have followed the instructions at > http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the > result. > > The only instance in which I diverged from the Wiki article is that I > used the Ubuntu-provided dovecot-antispam package (instead of compiling > the plug-in from source). > > I am trying to use the "mailtrain" back-end, per the above-cited Wiki > article, to call SpamAssassin's "sa-learn" utility for real-time learning. > > I am not using the SpamAssassin daemon (spamd). > > I'm about to throw in the towel on this plug-in, so any help would be > greatly appreciated. > > Thank you, > > -Ben I've never used this plugin but let offer up my home brewed script that I call from crontab: # Process and delete .Spam and .Ham Maildirs 11 */4 * * * /usr/local/sbin/maildir.sa-learn and /usr/local/sbin/maildir.sa-learn: #!/bin/bash MAILDIR=Maildir LS=/bin/ls RM=/bin/rm SU=/bin/su FIND=/bin/find SALEARN=/usr/bin/sa-learn for i in `$LS /home`; do prefs="/home/$i/.spamassassin/user_prefs" # [ ! ."$i" == ".bill" ] && continue echo $i spamdir="/home/$i/$MAILDIR/.SystemFolders.Spam" [ ! -d "$spamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/new" $i $FIND "$spamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --spam $spamdir/cur" $i $FIND "$spamdir/cur" -type f -delete hamdir="/home/$i/$MAILDIR/.SystemFolders.Ham" [ ! -d "$hamdir" ] && continue $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/new" $i $FIND "$hamdir/new" -type f -delete $SU -c "$SALEARN --prefspath=$prefs --ham $hamdir/cur" $i $FIND "$hamdir/cur" -type f -delete done With my setup each user has their own spamassassin user_prefs and bayes tokens. Bill From robert at schetterer.org Thu Sep 20 11:42:05 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 10:42:05 +0200 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: <505AC4B2.2040900@one.com> References: <505AC4B2.2040900@one.com> Message-ID: <505AD6DD.3020700@schetterer.org> Am 20.09.2012 09:24, schrieb Peter Mogensen: > Preventing download will prevent reading the mail famous last words *g -- Best Regards MfG Robert Schetterer From david at blue-labs.org Thu Sep 20 12:16:06 2012 From: david at blue-labs.org (David Ford) Date: Thu, 20 Sep 2012 05:16:06 -0400 Subject: [Dovecot] Prevent Download messages from server In-Reply-To: References: Message-ID: <505ADED6.7030204@blue-labs.org> do you mean to leave a copy of the email on the server so it can be read in multiple email clients? IMAP can do this and i think modern POP3 can. look for an account config option in your mail client to "leave mail on server". i think there is a setting in dovecot to prevent expunging of email but it has been years since i was researching this. -david On 09/20/2012 02:21 AM, Selcuk Yazar wrote: > Hi, > > can we prevent download messages from server user by user ? sme common used > mail's message must be remain at the server, but sometimes we download them > ? > > thanks in advance > From robert at schetterer.org Thu Sep 20 17:51:04 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 20 Sep 2012 16:51:04 +0200 Subject: [Dovecot] v2.1.10 released / indexer errors / lucene In-Reply-To: <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> References: <1347994347.13571.67.camel@hurina> <5059D5C1.500@schetterer.org> <70674664-8921-44D9-B843-0AEC15EA09B6@iki.fi> Message-ID: <505B2D58.3010301@schetterer.org> Am 19.09.2012 17:16, schrieb Timo Sirainen: > On 19.9.2012, at 17.25, Robert Schetterer wrote: > >> Am 18.09.2012 20:52, schrieb Timo Sirainen: >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz >>> http://dovecot.org/releases/2.1/dovecot-2.1.10.tar.gz.sig >>> >> >> Hi Timo, i see a few rare errors with lucene and/or indexer >> since update > > Hmm. But not before? I did one change to fts-lucene, but it wasn't supposed to change the existing behavior. > >> /lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out >> lucene-indexes) failed: Directory not empty >> >> Error: unlink_directory(/usr/local/virtual/.../.../lucene-indexes) >> failed: Directory not empty >> >> lucene-indexes: IndexReader::open() failed (#1): File does not exist > > The above errors are also from indexer-worker? I wonder how it gets to unlink_directory(). That shouldn't happen I think, at least I don't see by grepping the sources how it could, unless something is configured wrong (user is able to see lucene-indexes as a mailbox and is able to delete it). > >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: pure virtual >> method called >> Sep 19 14:46:19 mail01 dovecot: indexer-worker: Error: terminate called >> without an active exception >> >> Error: indexer failed to index mailbox INBOX >> >> should i care >> -- >> Best Regards >> MfG Robert Schetterer >> > update: the above errors didnt happen again today for one time this error comes up instead Error: Log synchronization error at seq=8,offset=28596 for /usr/local/virtual/...../...../dovecot.index: Extension header update without intro prefix -- Best Regards MfG Robert Schetterer From surreal at bsdmail.com Thu Sep 20 18:24:16 2012 From: surreal at bsdmail.com (Surreal) Date: Thu, 20 Sep 2012 23:24:16 +0800 Subject: [Dovecot] userdb missing driver Message-ID: <505B3520.10309@bsdmail.com> Good Day, I'm using dovecot 2.1.8 and this is my dovecot -n output: # 2.1.8: dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = domain.xxx auth_mechanisms = plain login auth_realms = domainone.xxx domaintwo.xxx disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver ---------------------------- This is my dovecot-sql.conf.ext connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix driver = mysql default_pass_scheme = MD5-CRYPT password_query = SELECT password,CONCAT('/mail/', maildir) AS userdb_home,\ '5000' AS userdb_uid, '5000' AS userdb_gid, allow_nets,\ concat('*:bytes=', quota) AS userdb_quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 user_query = SELECT CONCAT('/mail/', maildir) AS home, 'maildir:~/' as mail, '5000' AS uid, '5000' AS gid,\ concat('*:bytes=', quota) AS quota_rule\ FROM mailbox WHERE username='%u' AND domain='%d' AND active=1 #iterate_query = SELECT username AS user FROM mailbox ------------------------------ I'm sure i granted all privileges to daemonuser to postfix db. This is my first on dovecot so my knowledge is very minimal, all help possible is very much appreciated. Thanks From h.reindl at thelounge.net Thu Sep 20 18:43:55 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 20 Sep 2012 17:43:55 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3520.10309@bsdmail.com> References: <505B3520.10309@bsdmail.com> Message-ID: <505B39BB.3060308@thelounge.net> Am 20.09.2012 17:24, schrieb Surreal: > Good Day, > > I'm using dovecot 2.1.8 and this is my dovecot -n output: > > # 2.1.8: dovecot.conf > # OS: FreeBSD 9.0-STABLE i386 > > doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver > > connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix > driver = mysql no idea about BSD but on linux systems these are typically seperated packages dovecot-2.1.10-4.fc17.20120919.rh.x86_64 dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From ben at indietorrent.org Thu Sep 20 18:53:12 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 11:53:12 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505A5297.8050808@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> Message-ID: <505B3BE8.5050202@indietorrent.org> On 9/19/2012 7:17 PM, Ben Johnson wrote: > > > On 9/19/2012 6:19 PM, Tom Hendrikx wrote: >> On 19/09/12 23:41, Ben Johnson wrote: >>> >>> >>> On 9/11/2012 10:40 AM, Ben Johnson wrote: >>>> You nailed it, Ben. Thanks! >>>> >>>> I downloaded/installed the dovecot-dev package from the same maintainer >>>> and was able to build Antispam without issue. >>>> >>>> The hang-up now seems to be with my configuration. >>>> >>>> I'm seeing the following error in my email client whenever I attempt to >>>> move an email message from, say, Inbox to Junk folder: >>>> >>>> [SERVERBUG] BUG Unknown internal error. >>>> >>>> Nothing at all appears in syslog when this happens. >>>> >>>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam >>>> , to the letter, I believe. >>>> >>>> The relevant portions of my configuration are as follows: >>>> >>>> --------------------- >>>> protocol imap { >>>> mail_plugins = quota imap_quota antispam >>>> } >>>> >>>> plugin { >>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>> sieve = /var/vmail/%d/%n/.sieve >>>> >>>> antispam_backend = MAILTRAIN >>>> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh >>>> antispam_mail_spam = --spam >>>> antispam_mail_notspam = --ham >>>> antispam_spam_pattern_ignorecase = spam;junk >>>> } >>>> --------------------- >>>> >>>> The permissions and execute bit are set in accordance with other >>>> executables in the same directory: >>>> >>>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh" >>>> -rwxr-xr-x 1 root root 297 Sep 5 11:17 sa-learn-pipe.sh >>>> >>>> I would provide more log content, but the debug options seem to be gone >>>> in the forked, >= 2.0 version of this plug-in. >>>> >>>> Any ideas are very much appreciated. >>>> >>>> Thanks, >>>> >>>> -Ben >>> >>> Given that nobody shared any insight here, I decided to stand-up a >>> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs. >>> >>> While the error is different this time around, I am still unable to get >>> any variant of the dovecot-antispam plug-in working. >>> >>> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and >>> I use the above configuration, my syslog fills-up with the following: >>> >>> dovecot: imap-login: Login: user=, method=PLAIN, >>> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS >>> dovecot: master: Error: service(imap): child 2879 returned error 3 >>> >>> Surprisingly, I can find only two other mentions of "error 3" anywhere >>> on the Internet. Those mentions were from another user to this mailing list: >>> >>> http://www.dovecot.org/list/dovecot/2012-August/067787.html >>> http://www.dovecot.org/list/dovecot/2012-August/067814.html >>> >>> The lack of "error 3" ubiquity leads me to believe that I must be doing >>> something stupid. >>> >>> There's no question that this behavior is related to the >>> antispam-plugin, because when I commented-out the relevant lines in >>> Dovecot's configuration file, the error does not occur. >>> >>> I have followed the instructions at >>> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the >>> result. >>> >>> The only instance in which I diverged from the Wiki article is that I >>> used the Ubuntu-provided dovecot-antispam package (instead of compiling >>> the plug-in from source). >>> >>> I am trying to use the "mailtrain" back-end, per the above-cited Wiki >>> article, to call SpamAssassin's "sa-learn" utility for real-time learning. >>> >>> I am not using the SpamAssassin daemon (spamd). >>> >>> I'm about to throw in the towel on this plug-in, so any help would be >>> greatly appreciated. >>> >>> Thank you, >>> >>> -Ben >> >> There are 2 versions of the dovecot-antispam package. The original >> version was created by Johannes Berg, then died almost a silent death, >> and was forked by Eugene Paskevitch. Later Johannes revived his original >> version somewhat. >> >> The ubuntu package is based on the 'revived Johannes fork' [1], while >> the documentation on the dovecot wiki describes the 'Eugene fork'. [2] >> >> [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright >> [2] http://wiki2.dovecot.org/Plugins/Antispam >> >> Thus you'll need to choose which plugin you want, then follow the >> complete procedure for that plugin only. They both can do what you want, >> but building and configuration are slightly different. >> >> It would be nice if Johannes and Eugene would either agree on a single >> version of the plugin, or rename one of the forks so they don't get >> mixed up that easily. I'm not sure if that has been discussed (on- or >> off-list) in the past. >> >> FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the >> 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to >> upgrade to 2.1 yet, so no experience there. >> >> -- >> Tom >> > > Thank you for your timely response, Tom. Much appreciated. > > Your remarks clear-up a lot of the confusion for me. I didn't realize > that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did > I realize that this is the version that is packaged with Ubuntu 12.04. > > Given that I'm more-or-less forced to work with Ubuntu LTS, I would > prefer to use the OS's package management system wherever possible. I > suppose, by extension, that I would prefer to use the "revived Johannes > fork". > > I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but > am unable to get either to work the way I want (mailtrain back-end with > real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki > article). Shouldn't the basic tenets of each version of the plug-in apply? > > Where should I start, in the way of seeking support? Basically, I intend > to use the Johannes version, but I want to configure it in the manner > described in the Eugene version Wiki article. > > I've already described (rather exhaustively) what happens when I attempt > to use the Johannes version and adapt the Eugene version instructions. > > I'm curious to know what method you've used over the years with such > glowing success. Which back-end did you use with Dovecot 1/Johannes? How > about with Dovecot 2/Eugene? > > Thanks again, > > -Ben > Thanks again, Tom. After you set me straight on the various versions of the plug-in, I was able to synthesize the relevant documentation and make it all work on Ubuntu 12.04 LTS with Dovecot 2. The key was to understand the following: 1.) The Ubuntu repositories use the Johannes versions of the plug-in (for both Dovecot 1 and 2). Ubuntu 10.04 LTS (Dovecot 1) http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html Ubuntu 12.04 LTS (Dovecot 2) http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to the Eugene version, and that version's configuration directives differ from the Johannes version 2 directives. So, anyone attempting to follow the Wiki article on Ubuntu will need to adapt the directives accordingly (see documentation links above for equivalent directive names). 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki instructions lacks a shebang; until a proper shebang is added to the shell script, script execution will fail with "status 1". I added the standard shebang: #!/bin/sh Here are the configuration directives that work for me: plugin { antispam_debug_target = syslog antispam_backend = pipe antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_pipe_tmpdir = /tmp antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham } My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. I'm using the following directives, but when I move mail from Inbox to Junk, for example, I see no evidence that antispam is taking action. plugin { antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_mail_tmpdir = /tmp antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_mail_notspam = --ham } Dovecot writes nothing at all to syslog. No errors, no reference to antispam. And the message is moved successfully (no failure/error notice in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", attempts to create a log file in /tmp, which works with Antispam 2. But no such luck here; nothing is ever written to /tmp. Version 2 of the plug-in includes a verbose debugging mode, but it seems that no such feature is present in version 1. I noticed also that version 1 seems not to require a back-end to be specified explicitly (as in version 2). Is the back-end implied via the configuration directives that are present? Respectfully, -Ben From f.bonnet at esiee.fr Thu Sep 20 18:57:17 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 20 Sep 2012 17:57:17 +0200 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B39BB.3060308@thelounge.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> Message-ID: <505B3CDD.6@esiee.fr> On 09/20/2012 05:43 PM, Reindl Harald wrote: > Am 20.09.2012 17:24, schrieb Surreal: >> Good Day, >> >> I'm using dovecot 2.1.8 and this is my dovecot -n output: >> >> # 2.1.8: dovecot.conf >> # OS: FreeBSD 9.0-STABLE i386 >> >> doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver >> >> connect = host=127.0.0.1 port=3306 user=daemonuser password=password dbname=postfix >> driver = mysql > no idea about BSD but on linux systems these are typically > seperated packages > > dovecot-2.1.10-4.fc17.20120919.rh.x86_64 > dovecot-mysql-2.1.10-4.fc17.20120919.rh.x86_64 > > > if you install from FreeBSD ports you need to enable the MYSQL backend during configuration ( make config ) From tss at iki.fi Thu Sep 20 19:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 20 Sep 2012 18:01:05 +0200 Subject: [Dovecot] doveadm with multiple commands Message-ID: This has been in my TODO for a while and today I implemented it (but didn't dare to test it yet :) So the problem has been that you want to run for example: doveadm expunge -A mailbox Trash savedbefore 7d doveadm purge -A This is annoying because it scans through the users' mailboxes twice, wasting disk IO and CPU. So how to make it do the scan only once? The syntax I thought of today would be: doveadm multi [-A | -u wildcards] [ [...]] So the first two commands could be for example: doveadm multi -A : expunge mailbox Trash savedbefore 7d : purge The ':' character is the separator. Any other string could be used as well, for example "?SEP?" or whatever. Thoughts? Any better name for the command than "multi"? From horsley1953 at gmail.com Thu Sep 20 15:11:26 2012 From: horsley1953 at gmail.com (Tom Horsley) Date: Thu, 20 Sep 2012 08:11:26 -0400 Subject: [Dovecot] Sieve as mail client? Message-ID: <20120920081126.0627cd71@tomh> A slightly off topic question, but maybe someone here has seen something like this: I want a program that operates entirely on the client side of an IMAP server, and implements the sieve filtering language by doing the necessary client side operations to move mail on the IMAP server, expunge mail, etc. It would also act as a proxy between my real mail client and the IMAP server so I'd never see any mail that hasn't been filtered first. The benefits are obvious: I could finally get sieve filtering on my stupid corporate exchange mail server :-). From pixilla at macports.org Thu Sep 20 19:12:25 2012 From: pixilla at macports.org (Bradley Giesbrecht) Date: Thu, 20 Sep 2012 09:12:25 -0700 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. Is there a Johannes version 2? I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. http://johannes.sipsolutions.net/files/antispam.html Regards, Bradley Giesbrecht (pixilla) -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2763 bytes Desc: not available URL: From jerry at seibercom.net Thu Sep 20 19:12:42 2012 From: jerry at seibercom.net (Jerry) Date: Thu, 20 Sep 2012 12:12:42 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B3CDD.6@esiee.fr> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> Message-ID: <20120920121242.120102d6@scorpio> On Thu, 20 Sep 2012 17:57:17 +0200 Frank Bonnet articulated: > if you install from FreeBSD ports you need to enable > the MYSQL backend during configuration ( make config ) If you don't install Dovecot via the FreeBSD ports system, you are just asking for problems. As previously stated, do run "make config" and perhaps "make config-recursive" prior to actually attempting to build the port. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ben at indietorrent.org Thu Sep 20 19:33:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 12:33:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> <9C0F39AC-E0E2-4943-9D92-F5E079F68183@macports.org> Message-ID: <505B454D.6000402@indietorrent.org> On 9/20/2012 12:12 PM, Bradley Giesbrecht wrote: > > On Sep 20, 2012, at 8:53 AM, Ben Johnson wrote: > >> Thanks again, Tom. After you set me straight on the various versions of >> the plug-in, I was able to synthesize the relevant documentation and >> make it all work on Ubuntu 12.04 LTS with Dovecot 2. >> >> The key was to understand the following: >> >> 1.) The Ubuntu repositories use the Johannes versions of the plug-in >> (for both Dovecot 1 and 2). >> >> Ubuntu 10.04 LTS (Dovecot 1) >> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html >> >> Ubuntu 12.04 LTS (Dovecot 2) >> http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html >> >> 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to >> the Eugene version, and that version's configuration directives differ >> from the Johannes version 2 directives. > > Is there a Johannes version 2? Thanks for your reply, Bradley. That seems to be the implication here: http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html Johannes is listed as an author, the package name is dovecot-antispam_2.0+20120225, and the manpage is nearly identical to that of version 1 at http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html . > I understood that Johannes dropped support at dovecot 2.x and that is why Eugene's version exists. > >> So, anyone attempting to follow >> the Wiki article on Ubuntu will need to adapt the directives accordingly >> (see documentation links above for equivalent directive names). >> >> 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki >> instructions lacks a shebang; until a proper shebang is added to the >> shell script, script execution will fail with "status 1". I added the >> standard shebang: >> >> #!/bin/sh >> >> Here are the configuration directives that work for me: >> >> plugin { >> antispam_debug_target = syslog >> antispam_backend = pipe >> antispam_signature = X-DSPAM-Signature >> antispam_signature_missing = error >> antispam_spam_pattern_ignorecase = SPAM;JUNK >> antispam_pipe_tmpdir = /tmp >> antispam_pipe_program = /usr/bin/sa-learn-pipe.sh >> antispam_pipe_program_spam_arg = --spam >> antispam_pipe_program_notspam_arg = --ham >> } >> >> My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > If I were implementing Johannes antispam plugin with Dovecot 1.2.9 I would follow Johannes documentation. > http://johannes.sipsolutions.net/files/antispam.html This documentation seems to apply to the Dovecot 2 version; the antispam_backend directive doesn't seem to exist in version 1, at least according to the official Ubuntu package documentation (cited above). In fact, this appears to be the exact document that is used for the Ubuntu package for Dovecot 2 (also cited above). > > Regards, > Bradley Giesbrecht (pixilla) > From surreal at bsdmail.com Thu Sep 20 19:35:08 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 00:35:08 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <20120920121242.120102d6@scorpio> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> Message-ID: <505B45BC.8020109@bsdmail.com> On 9/21/2012 12:12 AM, Jerry wrote: > On Thu, 20 Sep 2012 17:57:17 +0200 > Frank Bonnet articulated: > >> if you install from FreeBSD ports you need to enable >> the MYSQL backend during configuration ( make config ) > If you don't install Dovecot via the FreeBSD ports system, you are just > asking for problems. As previously stated, do run "make config" and > perhaps "make config-recursive" prior to actually attempting to build > the port. > I did, MySQL is ticked when i build dovecot2 from /usr/ports/mail/dovecot2/ From ml at cibest.com Thu Sep 20 20:04:27 2012 From: ml at cibest.com (Cibest ML) Date: Thu, 20 Sep 2012 19:04:27 +0200 Subject: [Dovecot] Wrong quota calculation In-Reply-To: <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> References: <5049B472.8060407@cibest.com> <20120907233844.GA12996@daniel.localdomain> <504B5F2E.7070202@cibest.com> <52065B76-BD51-412D-9948-7648BC9D1905@iki.fi> <50504ED3.5030601@cibest.com> <5059B774.6060808@cibest.com> <79E87B01-B764-47CB-8B15-84160412EF4C@iki.fi> Message-ID: <505B4C9B.5080708@cibest.com> Hi Timo, I've finally solved the issue. As usual, thinking out of the box helped :) In fact, it was conf related but not where it was expected. There was an autocreate = Inbox & autosubscribe = Inbox in 90-plugin.conf Having this configuration seems to double count the Inbox quota during quota recalc (but not during normal IMAP operations like moving or receiving a message). Might be some kind of bug somewhere in recalc. The strange thing is that another more simple dovecot system we have (no nfs, all imap/smtp/filtering operations done on the server) also has this plugin conf but doesn't count quota twice. Anyway, after running a small script to recalc all the quota, everything is now fine. Thanks for you help -- Regards S?bastien On 19/09/2012 14:47, Timo Sirainen wrote: > On 19.9.2012, at 15.15, Cibest ML wrote: > >> There's one quite strange thing we've seen. Display the subscription windows in Thunderbird and unchecking INBOX then shows two entries. "INBOX" that isn't selected" and "Inbox" that is selected and cannot been unselected. Checking "INBOX" again makes "Inbox" disappear (subscription window need to be closed & reopened after each action). > No idea. INBOX is treated case-insensitively by IMAP protocol. > >> I'm not sure that what we see in Thunderbird is related to the problem but who knows. Any help is of course welcome as we're out of ideas :) > Create a test server. Make your Dovecot config minimal. Test with another quota backend. Test with local filesystem. See what change breaks the quota calculation. It is somehow related to your current config, since it's not broken for everybody. > From daryl at isletech.net Thu Sep 20 21:42:05 2012 From: daryl at isletech.net (Daryl Richards) Date: Thu, 20 Sep 2012 14:42:05 -0400 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B45BC.8020109@bsdmail.com> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> Message-ID: <505B637D.4080901@isletech.net> On 12-09-20 12:35 PM, Surreal wrote: > On 9/21/2012 12:12 AM, Jerry wrote: >> On Thu, 20 Sep 2012 17:57:17 +0200 >> Frank Bonnet articulated: >> >>> if you install from FreeBSD ports you need to enable >>> the MYSQL backend during configuration ( make config ) >> If you don't install Dovecot via the FreeBSD ports system, you are just >> asking for problems. As previously stated, do run "make config" and >> perhaps "make config-recursive" prior to actually attempting to build >> the port. >> > I did, MySQL is ticked when i build dovecot2 from > /usr/ports/mail/dovecot2/ If you do dovecot --build-options, does it list mysql? root at mail:~# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql -- Daryl Richards Isle Technical Services Inc. From daniel.parthey at informatik.tu-chemnitz.de Thu Sep 20 23:17:41 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 20 Sep 2012 22:17:41 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <20120920201741.GA9646@daniel.localdomain> Timo Sirainen wrote: > doveadm multi [-A | -u wildcards] [ [...]] > > Thoughts? Would be nice if this also worked through doveadm director proxy and executes the command sequence on the correct backend host for each user. As command name I could also think of "doveadm sequence", which implies the commands being executed in serial order. Regards Daniel -- https://plus.google.com/103021802792276734820 From ben at indietorrent.org Fri Sep 21 00:09:17 2012 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 20 Sep 2012 17:09:17 -0400 Subject: [Dovecot] Trouble implementing Antispam plug-in for Dovecot In-Reply-To: <505B3BE8.5050202@indietorrent.org> References: <5047D049.8080004@indietorrent.org> <50487684.8060404@Media-Brokers.com> <5048E3EA.80009@indietorrent.org> <504E3D29.10102@indietorrent.org> <20120911011456.GA3908@anubis.morrow.me.uk> <504F4D5D.8040207@indietorrent.org> <505A3C12.7050207@indietorrent.org> <505A44DB.6020200@whyscream.net> <505A5297.8050808@indietorrent.org> <505B3BE8.5050202@indietorrent.org> Message-ID: <505B85FD.4000604@indietorrent.org> On 9/20/2012 11:53 AM, Ben Johnson wrote: > Thanks again, Tom. After you set me straight on the various versions of > the plug-in, I was able to synthesize the relevant documentation and > make it all work on Ubuntu 12.04 LTS with Dovecot 2. > > The key was to understand the following: > > 1.) The Ubuntu repositories use the Johannes versions of the plug-in > (for both Dovecot 1 and 2). > > Ubuntu 10.04 LTS (Dovecot 1) > http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html > > Ubuntu 12.04 LTS (Dovecot 2) > http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html > > 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to > the Eugene version, and that version's configuration directives differ > from the Johannes version 2 directives. So, anyone attempting to follow > the Wiki article on Ubuntu will need to adapt the directives accordingly > (see documentation links above for equivalent directive names). > > 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki > instructions lacks a shebang; until a proper shebang is added to the > shell script, script execution will fail with "status 1". I added the > standard shebang: > > #!/bin/sh > > Here are the configuration directives that work for me: > > plugin { > antispam_debug_target = syslog > antispam_backend = pipe > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_pipe_tmpdir = /tmp > antispam_pipe_program = /usr/bin/sa-learn-pipe.sh > antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > } > > My final goal is to get the version 1 plug-in working with Dovecot 1.2.9. > > I'm using the following directives, but when I move mail from Inbox to > Junk, for example, I see no evidence that antispam is taking action. > > plugin { > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam_pattern_ignorecase = SPAM;JUNK > antispam_mail_tmpdir = /tmp > antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh > antispam_mail_spam = --spam > antispam_mail_notspam = --ham > } > > Dovecot writes nothing at all to syslog. No errors, no reference to > antispam. And the message is moved successfully (no failure/error notice > in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh", > attempts to create a log file in /tmp, which works with Antispam 2. But > no such luck here; nothing is ever written to /tmp. > > Version 2 of the plug-in includes a verbose debugging mode, but it seems > that no such feature is present in version 1. > > I noticed also that version 1 seems not to require a back-end to be > specified explicitly (as in version 2). Is the back-end implied via the > configuration directives that are present? > > Respectfully, > > -Ben Just a follow-up to report that I finally have the Antispam plug-in (the Johannes version) -- with the mailtrain/pipe back-end -- working as expected in both of the following environments: - Ubuntu 10.04 with Dovecot 1.2.9 - Ubuntu 12.04 with Dovecot 2.0.19 In addition to the points that I mentioned in my previous note, a crucial step in the debugging process was to modify the "sa-learn-pipe.sh" script to produce more verbose output. Unlike Antispam 2, Antispam 1 lacks a verbose debugging mode, so when the plug-in does not work as expected, tracking-down the root cause can be much more difficult. Here's the shell script from http://wiki2.dovecot.org/Plugins/Antispam , with my modifications: http://pastebin.com/igTXCYQF After making these modifications to the shell script, I was able to determine the cause of the failure in Dovecot 1.2.9. In fact, I was incorrect earlier when I stated that version 2 was working; the same problem applied there, too. The "vmail" user, as whom the script is executed, lacks the permissions required to change the Bayes token database (this is a "good thing"). An excerpt from the custom log file: 29722-start (--spam) Calling (as user vmail) '/usr/bin/sa-learn --spam /tmp/sendmail-msg-29722.txt' bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 493. Learned tokens from 0 message(s) (1 message(s) examined) 29722-end The only means by which I was able to work around this issue -- without changing filesystem permissions, users, groups, or otherwise compromising system security -- was to migrate from the standard flat-file Bayes database format to MySQL. (The guide at http://www.starbridge.org/spip/spip.php?article15 had everything I needed, and it only took me a few minutes to implement.) After making that change, everything began working as expected; here's a sample custom log entry: 30565-start (--ham) Calling (as user vmail) '/usr/bin/sa-learn --ham /tmp/sendmail-msg-30565.txt' Learned tokens from 1 message(s) (1 message(s) examined) 30565-end In any event, this concludes the saga, and I must say, I'm thrilled to have it all working. Thanks again to everyone who contributed valuable input and pointed me in the right direction. And thanks to Johannes and Eugene (and everyone else who contributed to Antispam) for solving a major hurdle in Bayes DB training. -Ben From surreal at bsdmail.com Fri Sep 21 04:35:22 2012 From: surreal at bsdmail.com (Surreal) Date: Fri, 21 Sep 2012 09:35:22 +0800 Subject: [Dovecot] userdb missing driver In-Reply-To: <505B637D.4080901@isletech.net> References: <505B3520.10309@bsdmail.com> <505B39BB.3060308@thelounge.net> <505B3CDD.6@esiee.fr> <20120920121242.120102d6@scorpio> <505B45BC.8020109@bsdmail.com> <505B637D.4080901@isletech.net> Message-ID: <505BC45A.1010003@bsdmail.com> On 9/21/2012 2:42 AM, Daryl Richards wrote: > On 12-09-20 12:35 PM, Surreal wrote: >> On 9/21/2012 12:12 AM, Jerry wrote: >>> On Thu, 20 Sep 2012 17:57:17 +0200 >>> Frank Bonnet articulated: >>> >>>> if you install from FreeBSD ports you need to enable >>>> the MYSQL backend during configuration ( make config ) >>> If you don't install Dovecot via the FreeBSD ports system, you are just >>> asking for problems. As previously stated, do run "make config" and >>> perhaps "make config-recursive" prior to actually attempting to build >>> the port. >>> >> I did, MySQL is ticked when i build dovecot2 from >> /usr/ports/mail/dovecot2/ > If you do dovecot --build-options, does it list mysql? > > root at mail:~# dovecot --build-options > Build options: ioloop=kqueue notify=kqueue ipv6 openssl > io_block_size=8192 > Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw > SQL drivers: mysql > Passdb: checkpassword pam passwd passwd-file sql > Userdb: checkpassword nss passwd prefetch passwd-file sql > At first i have noticed that i dont have mysql when i did --build-options, so i tried to deinstall and reinstall dovecot2 port and now i have this also: hostname# dovecot --build-options Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: mysql Passdb: checkpassword pam passwd passwd-file sql Userdb: checkpassword nss passwd prefetch passwd-file sql But i still keep getting the same error hostname# service dovecot start doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver doveconf: Error: userdb is missing driver doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver Starting dovecot. doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot Still the same output for doveconf -n hostname# doveconf -n # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_default_realm = ren.wf auth_mechanisms = plain login auth_realms = ren.wf graphics.re disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 lda_mailbox_autocreate = yes mail_gid = 5000 mail_location = maildir:~/ mail_plugins = quota expire mail_log notify mail_temp_dir = /var/tmp mail_uid = 5000 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota2_rule = *:storage=3G quota_rule = *:storage=3G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=75%% quota-warning 75 %u } sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } doveconf: Error: userdb is missing driver protocol imap { mail_max_userip_connections = 3 mail_plugins = quota expire mail_log notify imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 2 } doveconf: Fatal: Error in configuration file /usr/local/etc/dovecot/dovecot.conf: userdb is missing driver From tss at iki.fi Fri Sep 21 08:28:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 21 Sep 2012 07:28:57 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <20120920201741.GA9646@daniel.localdomain> References: <20120920201741.GA9646@daniel.localdomain> Message-ID: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> On 20.9.2012, at 22.17, Daniel Parthey wrote: > Timo Sirainen wrote: >> doveadm multi [-A | -u wildcards] [ [...]] >> >> Thoughts? > > Would be nice if this also worked through doveadm director proxy > and executes the command sequence on the correct backend host for > each user. Yes, it's going to. > As command name I could also think of "doveadm sequence", which > implies the commands being executed in serial order. Hmm. Maybe. From alec at alec.pl Fri Sep 21 11:23:02 2012 From: alec at alec.pl (A.L.E.C) Date: Fri, 21 Sep 2012 10:23:02 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: Message-ID: <505C23E6.2070606@alec.pl> On 09/20/2012 06:01 PM, Timo Sirainen wrote: > Thoughts? Any better name for the command than "multi"? How about 'execute' or 'exec'. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From tomas at blueboard.cz Fri Sep 21 12:32:37 2012 From: tomas at blueboard.cz (=?ISO-8859-2?Q?Tom=E1=B9_Randa?=) Date: Fri, 21 Sep 2012 11:32:37 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? Message-ID: <505C3435.30601@blueboard.cz> Hello, I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or dovecot<>freebsd bug? I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over 1000 processes with kernel message: maxproc limit exceeded by uid 89 Could anybody help? Many thanks Tomas My system have following settings: FreeBSD 9.0 / AMD64 Dovecot 2.1.8 kern.maxproc: 12288 kern.maxfilesperproc: 36864 kern.maxprocperuid: 8192 no limit for uid 89: # limit cputime unlimited filesize unlimited datasize 33554432 kbytes stacksize 524288 kbytes coredumpsize unlimited memoryuse unlimited vmemoryuse unlimited descriptors 36864 memorylocked unlimited maxproc 8192 sbsize unlimited swapsize unlimited My dovecot.conf: # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE amd64 auth_mechanisms = plain login digest-md5 cram-md5 default_client_limit = 2048 default_process_limit = 2048 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /data/logfiles/dovecot/dovecot-info.log last_valid_gid = 89 last_valid_uid = 89 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Mail Toaster (Dovecot) ready. mail_location = maildir:~/Maildir mail_plugins = " quota" mail_privileged_group = mail maildir_broken_filename_sizes = yes passdb { driver = vpopmail } plugin { quota = maildir quota_rule = Trash:ignore sieve = ~/.sieve/dovecot.sieve sieve_dir = ~/.sieve } protocols = imap pop3 sieve service anvil { client_limit = 6147 } service auth { client_limit = 8192 unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } } service imap-login { process_limit = 2048 service_count = 1 } service imap { client_limit = 1 process_limit = 2048 } service managesieve { process_limit = 2048 } service pop3-login { process_limit = 2048 service_count = 1 } service pop3 { client_limit = 1 process_limit = 2048 } shutdown_clients = no ssl_cert = object mailstore In-Reply-To: <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <3270EFD6-64E2-45E0-B88C-DD240B57BBD1@iki.fi> Message-ID: On 17 September 2012 13:57, Timo Sirainen wrote: > On 14.9.2012, at 18.16, Damien Churchill wrote: > >> On 14 September 2012 15:59, Timo Sirainen wrote: >>> On 14.9.2012, at 17.46, Damien Churchill wrote: >>> >>>> I was wondering what would be entailed in modifying dovecot 2.2 to >>>> support storing mail in an object store. I've seen a few mails dotted >>>> around in the ML history about supporting such a thing and seen it's >>>> basically dependant upon some changes in lib-storage to support >>>> writing messages without locking. Is this still the case? >>> >>> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use? >>> >> >> That's good to hear :) I've been evaluating RADOS as an object store, >> which is similar to S3. Although any distributed storage would be >> great. I'd be more than happy to help code it! > > I think I'll first have to get started with it to see if there are some parts that are easy to give to you. First I'll at least need to do some refactoring to dbox code and lib-fs code. I'm planning on making the generic parts of it be part of Dovecot releases, but I haven't yet fully decided which backends should be.. > >>> The generic idea is: >>> - only one server accesses one user simultaneously >>> - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage >>> - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync >>> - support high latency: asynchronous reads/writes. prefetch mail bodies. >> >> I'm assuming that the director would be used in order to distribute >> connections to the same server, so it's only within a local instance >> of dovecot you'd need to be aware of what currently has a connection >> open for that user? > > Right. Probably some new process that can do the work of downloading/uploading/deleting index files as needed. That's actually a clearly separate task that you could do? :) > Sounds good! I'll spend some time digging through the source code getting familiar. >> How are you planning on handling the situation where say node X dies >> and hasn't uploaded the latest index file? Would that result in >> missing messages from the mailbox when accessed by another node, or is >> the local index intended to be more of a write-through cache? > > No messages get ever lost. Recent flag changes and expunges may get lost, at least until the original node comes back up and dsync merges the changes. Idea was that when downloading index a flag on the object storage is set for the user that it's being accessed, and removed after the user is disconnected and index is uploaded back. If index downloader already sees that the flag is set it will run some kind of a recovery process to find any messages that were uploaded but not indexed. (The message bodies are always immediately uploaded to object storage.) Part of me thinks making this configurable might be a good idea depending upon what the installation is trying to achieve. Since the recovery process will need to be implemented regardless allowing the user to configure dovecot to perform a write to both the local and object index when a flag or something else is modified. Another thought occurs to me, when using the LDA how will that be able to update the index upon delivery of a new message if another node is currently accessing the mailbox? From h.reindl at thelounge.net Fri Sep 21 12:52:50 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 11:52:50 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505C38F2.40402@thelounge.net> Am 21.09.2012 11:32, schrieb Tom?? Randa: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but hard limit is 8192 per user in box. I tried > everything, including modifying startup script of dovecot to set ulimit -u 8192. Could it be some dovecot bug or > dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap clients in one process, but still I am over > 1000 processes with kernel message: no idea about BSD but your config are a total of up to 10.240 PROCESSES one process has MUCH MORE than one file-handle i have ONE imap-login process with 572 file-handles your configuration eats up to 5 Mio. file-handles maybe you running out of OS ressources 1000 prcoesses are up to 500000 file handles for one service [root at mail:~]$ ps aux | grep imap-login | wc -l 2 [root at mail:~]$ lsof | grep imap-l | wc -l 572 ___________________ for imaplogin / pop3-login as example you do not need a PROCESS per connection service_count = 0 process_min_avail = 1 process_limit = 10 client_limit = 200 this can handle 2000 connections with up to 10 processes > service imap-login { > process_limit = 2048 > service_count = 1 > } > service imap { > client_limit = 1 > process_limit = 2048 > } > service managesieve { > process_limit = 2048 > } > service pop3-login { > process_limit = 2048 > service_count = 1 > } > service pop3 { > client_limit = 1 > process_limit = 2048 > } -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From trashcan at odo.in-berlin.de Fri Sep 21 14:48:45 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 21 Sep 2012 13:48:45 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: On 2012-09-21 10:23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. doveadm execall [...] Regards, Michael From matthias at couch-blog.de Fri Sep 21 16:09:06 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:09:06 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? Message-ID: <505C66F2.7060704@couch-blog.de> Hello! Could it be, that http://xi.rename-it.nl is down? Thanks, Matthias From h.reindl at thelounge.net Fri Sep 21 16:22:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:22:03 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C66F2.7060704@couch-blog.de> References: <505C66F2.7060704@couch-blog.de> Message-ID: <505C69FB.3040402@thelounge.net> Am 21.09.2012 15:09, schrieb Matthias: > Hello! > > Could it be, that http://xi.rename-it.nl is down? not from vienna but what the hell has this question to search on the dovecot-list? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Fri Sep 21 16:27:37 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 21 Sep 2012 15:27:37 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132737.GX9342@charite.de> * Reindl Harald : > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? Isn't that where the sieve extension is located at? http://xi.rename-it.nl/debian/dists/unstable-auto/ -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mailinglist at darac.org.uk Fri Sep 21 16:27:47 2012 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 21 Sep 2012 14:27:47 +0100 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C69FB.3040402@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> Message-ID: <20120921132747.GA23967@darac.org.uk> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: > > > Am 21.09.2012 15:09, schrieb Matthias: > > Hello! > > > > Could it be, that http://xi.rename-it.nl is down? > > not from vienna > > but what the hell has this question to search > on the dovecot-list? > The mentioned site is a host for Prebuilt Binaries: http://wiki2.dovecot.org/PrebuiltBinaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From h.reindl at thelounge.net Fri Sep 21 16:40:51 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 21 Sep 2012 15:40:51 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <20120921132747.GA23967@darac.org.uk> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> Message-ID: <505C6E63.6020608@thelounge.net> Am 21.09.2012 15:27, schrieb Darac Marjal: > On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >> >> >> Am 21.09.2012 15:09, schrieb Matthias: >>> Hello! >>> >>> Could it be, that http://xi.rename-it.nl is down? >> >> not from vienna >> >> but what the hell has this question to search >> on the dovecot-list? >> > > The mentioned site is a host for Prebuilt Binaries: > http://wiki2.dovecot.org/PrebuiltBinaries ok, this should have been mentioned because only few people understand "Welkom op de website van Rename-it, uw partner in systeembeheer en internetoplossingen." * most people using packages from their distribution * the rest is compiling from source * one or two others are using foreign binaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From sb at dod.no Fri Sep 21 16:53:29 2012 From: sb at dod.no (Steinar Bang) Date: Fri, 21 Sep 2012 15:53:29 +0200 Subject: [Dovecot] doveadm with multiple commands References: Message-ID: <87mx0jbhye.fsf@dod.no> >>>>> Timo Sirainen : > Thoughts? How about: a) Dropping into a command prompt where it's possible to give commands? b) Making it possible to feed commands through a pipe (ie reading lines from stdin)? From matthias at couch-blog.de Fri Sep 21 16:55:48 2012 From: matthias at couch-blog.de (Matthias) Date: Fri, 21 Sep 2012 15:55:48 +0200 Subject: [Dovecot] http://xi.rename-it.nl down? In-Reply-To: <505C6E63.6020608@thelounge.net> References: <505C66F2.7060704@couch-blog.de> <505C69FB.3040402@thelounge.net> <20120921132747.GA23967@darac.org.uk> <505C6E63.6020608@thelounge.net> Message-ID: <505C71E4.6070105@couch-blog.de> Am 21.09.2012 15:40, schrieb Reindl Harald: > > Am 21.09.2012 15:27, schrieb Darac Marjal: >> On Fri, Sep 21, 2012 at 03:22:03PM +0200, Reindl Harald wrote: >>> >>> Am 21.09.2012 15:09, schrieb Matthias: >>>> Hello! >>>> >>>> Could it be, that http://xi.rename-it.nl is down? >>> not from vienna >>> >>> but what the hell has this question to search >>> on the dovecot-list? >>> >> The mentioned site is a host for Prebuilt Binaries: >> http://wiki2.dovecot.org/PrebuiltBinaries > ok, this should have been mentioned because only few > people understand "Welkom op de website van Rename-it, > uw partner in systeembeheer en internetoplossingen." > > * most people using packages from their distribution > * the rest is compiling from source > * one or two others are using foreign binaries > Oh now all is great, thanks for small help ;) I test a little bit with a small sieve skript, to filter my mails on my mailserver with dovecot 2.1. Now it just works. Sorry if the question was on the wrong mailinglist. From alessio at skye.it Fri Sep 21 17:13:15 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 21 Sep 2012 16:13:15 +0200 Subject: [Dovecot] key -> object mailstore In-Reply-To: <1347680376.5717.61.camel@maclinux> References: <92DBD8A1-85BB-45A8-A2A5-09FF8664418C@iki.fi> <1347680376.5717.61.camel@maclinux> Message-ID: <505C75FB.4080006@skye.it> Il 15/09/2012 05:39, Jeff Gustafson ha scritto: > On Fri, 2012-09-14 at 17:59 +0300, Timo Sirainen wrote: >> I've a whole new design for it and I was planning on implementing it >> for v2.2. Do you want to help coding it? :) Which storage would you >> want to use? >> >> The generic idea is: >> - only one server accesses one user simultaneously >> - index files are copied from object storage to local filesystem and >> accessed there, once in a while uploaded back to object storage >> - if user is accessed from two servers because of some bug/split >> brain/something, the changes are merged using dsync >> - support high latency: asynchronous reads/writes. prefetch mail bodies. > With this system, would the read/write ultimately go to a normal OS > file function? If it is a file function, could this be used with a > system like glusterfs, ceph, etc? The other option would be to write it > against a object store client library and bypass the normal file > functions. > > ...Jeff > Also other users are talking about Ceph and Dovecot http://www.mail-archive.com/ceph-devel at vger.kernel.org/msg07345.html -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stan at hardwarefreak.com Sat Sep 22 00:16:50 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 21 Sep 2012 16:16:50 -0500 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505CD942.7000104@hardwarefreak.com> On 9/21/2012 4:32 AM, Tom?? Randa wrote: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. What is your value for kern.maxusers? Did you try increasing it? Note in the 2nd paragraph below the relationship between kern.maxusers and process limit. From what you describe it would seem you have a process limit of 1044, thus a kern.maxusers value of 64. Considering your manual setting of 8192 processes is apparently being ignored, it would seem the kern.maxusers value is causing it to be overridden. From: http://www.pl.freebsd.org/doc/handbook/configtuning-kernel-limits.html As of FreeBSD 4.5, kern.maxusers is automatically sized at boot based on the amount of memory available in the system, and may be determined at run-time by inspecting the value of the read-only kern.maxusers sysctl. Some sites will require larger or smaller values of kern.maxusers and may set it as a loader tunable; values of 64, 128, and 256 are not uncommon. We do not recommend going above 256 unless you need a huge number of file descriptors; many of the tunable values set to their defaults by kern.maxusers may be individually overridden at boot-time or run-time in /boot/loader.conf (see the loader.conf(5) man page or the /boot/defaults/loader.conf file for some hints) or as described elsewhere in this document. Systems older than FreeBSD 4.4 must set this value via the kernel config(8) option maxusers instead. In older releases, the system will auto-tune maxusers for you if you explicitly set it to 0[1]. When setting this option, you will want to set maxusers to at least 4, especially if you are using the X Window System or compiling software. The reason is that the most important table set by maxusers is the maximum number of processes, which is set to 20 + 16 * maxusers, so if you set maxusers to 1, then you can only have 36 simultaneous processes, including the 18 or so that the system starts up at boot time and the 15 or so you will probably create when you start the X Window System. Even a simple task like reading a manual page will start up nine processes to filter, decompress, and view it. Setting maxusers to 64 will allow you to have up to 1044 simultaneous processes, which should be enough for nearly all uses. If, however, you see the dreaded proc table full error when trying to start another program, or are running a server with a large number of simultaneous users (like ftp.FreeBSD.org), you can always increase the number and rebuild. -- Stan From durket at highwire.stanford.edu Sat Sep 22 00:54:02 2012 From: durket at highwire.stanford.edu (Michael Durket) Date: Fri, 21 Sep 2012 14:54:02 -0700 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? Message-ID: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? From robert at schetterer.org Sat Sep 22 08:46:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 07:46:15 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> Message-ID: <505D50A7.3090003@schetterer.org> Am 21.09.2012 23:54, schrieb Michael Durket: > I'm seeing this periodically in my logs for a specific user (followed by a backtrace which I've deleted from this post): > > Panic: file dovecot-1.2.11/src/lib-index/mail-transaction-log-view.c: line 290 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || max_file_seq != view->head->hdr.file_seq || max_file_offset != (uoff_t)-1 || min_file_offset <= view->head->sync_offset) > > Is there something I can do to work around this problem for the user, or is this something new and I need to follow the instructions at http://dovecot.org/bugreport.html to send in a bug report? > you should upgrade to recent stable version 2.1.x -- Best Regards MfG Robert Schetterer From tlhackque at yahoo.com Sat Sep 22 15:04:22 2012 From: tlhackque at yahoo.com (tlhackque) Date: Sat, 22 Sep 2012 08:04:22 -0400 Subject: [Dovecot] Spurious " Renaming not supported across conflicting directory permissions" Message-ID: <505DA946.90409@yahoo.com> Dovecot 2.1.10 Client = Thunderbird. Local disks. mbox format. Attempted to rename a folder, failed with: CANNOT Renaming not supported across conflicting directory permissions I don't understand why this should happen. o I was renaming within a directory - the attempt was to rename "Domain Names &- SSL Certificates" to "Domain Names". No directory was specified o The reason the directory has g=s is for dovecot - so that the 'mail' group (12) will stick to any files created. I was getting more understandable permission failures before I added it. Since 's' doesn't apply to files, I don't see why this should be viewed as a conflict. o If I chmod g-s the directory, the command succeeds. But then I'm back to file ownership problems... drwxrws--- 4 4503 12 4096 Sep 22 06:17 F/N/ -rw-rw---- 1 4503 12 81530 Sep 22 06:17 F/N/Domain Names &- SSL Certificates Sequence from TB's trace (crlfs for clarity): 1140[11ebcf00]: 121e8c00:imap.example.com:A:SendData: 66 rename "F/N/Domain Names &- SSL Certificates" "F/N/Domain Names" 1140[11ebcf00]: ReadNextLine [stream=12135a28 nb=80 needmore=0] 1140[11ebcf00]: 121e8c00:imap.example.com:A:CreateNewLineFromSocket: 66 NO [CANNOT] Renaming not supported across conflicting directory permissions 9592[c7d6840]: ReadNextLine [stream=ab687e8 nb=22 needmore=0] Config: # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.22.14-72.fc6 i686 Fedora Core release 6 (Zod) first_valid_gid = 4000 first_valid_uid = 4000 hostname = smtp.example.com login_greeting = Dovecot ready. Unauthorized access is prohibited. mail_location = mbox:~/mail:INBOX=/var/mail/%n mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { driver = pam } service imap-login { inet_listener imap { address = imap.v4.example.com imap.v6.example.com port = 143 } inet_listener imaps { address = imap.v4.example.com imap.v6.example.com port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { address = pop.v4.example.com pop.v6.example.com port = 110 } inet_listener pop3s { address = pop.v4.example.com pop.v6.example.com port = 995 ssl = yes } } ssl_ca = Hi, running in problems with KMail I decided to switch to dovecot-imap & thunderbird on my kubuntu 12.04 64 bit (so I can read my imap-folder-mails with KMail again if it gets more stable) What I did: - I wrote a python script to convert my KMail mails folder to a thunderbird mailfolder (directory tree containing mbox files and *.sbd folders) - I created a fake pop mail account in thunderbird and imported the whole mail-structure (i just copied it) - I create a local imap account ( so I have a ~/Maildir dovecot folder in my user dir) - In /etc/dovecot: 1. /etc/dovecot/10-mail.conf (line 10) I set: mail_location = maildir:~/Maildir:LAYOUT=fs 2. /etc/dovecot/20-imap.conf (line 57) I set: imap_client_workarounds = tb-extra-mailbox-sep Now I want to copy the whole mailstructure from my imported fake pop mailfolder to the dovecot imap folder. What happens: Lets say my source structure is mailfolder1 + mail1 + mail2 + subMailFolder2 + mail1-in-subMailFolder2 + mail2-in-subMailFolder2 + subSubMailFolder3 + mail1-in-subMailFolder3 + mail2-in-subMailFolder3 I thunderbird (version 15) I drag and drop my mailFolder1 to the imap folder and I get: mailfolder1 + mail1 + mail2 + subMailFolder2 + subSubMailFolder3 as you See only the mails of the *TOP* folder are copied. All subfolders are created but they contain NO MAILS. Did somebody experience the same problem??? Otherwise I will write a python script to do that (I dont hope to need to write a mailclient in python ;-) ) Any hint is greatly welcome. Anton From me at junc.org Sat Sep 22 16:57:54 2012 From: me at junc.org (Benny Pedersen) Date: Sat, 22 Sep 2012 15:57:54 +0200 Subject: [Dovecot] =?utf-8?q?Dovecot_1=2E2=2E11_panic_in_mail-transaction-?= =?utf-8?q?log-view=3F?= In-Reply-To: <505D50A7.3090003@schetterer.org> References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: Robert Schetterer skrev den 2012-09-22 07:46: > you should upgrade to recent stable version 2.1.x and this is a simple change from 1.x to 2.x :( i have never a panic in 1.x here From robert at schetterer.org Sat Sep 22 17:03:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sat, 22 Sep 2012 16:03:12 +0200 Subject: [Dovecot] Dovecot 1.2.11 panic in mail-transaction-log-view? In-Reply-To: References: <47CF5D1E-D0C2-43C6-9535-6AF0F8F33518@highwire.stanford.edu> <505D50A7.3090003@schetterer.org> Message-ID: <505DC520.9050505@schetterer.org> Am 22.09.2012 15:57, schrieb Benny Pedersen: > Robert Schetterer skrev den 2012-09-22 07:46: > >> you should upgrade to recent stable version 2.1.x > > and this is a simple change from 1.x to 2.x :( depends on how complex your setup is youre should really try upgrade http://wiki.dovecot.org/Upgrading/1.2 http://wiki2.dovecot.org/Upgrading/2.0 > > i have never a panic in 1.x here > yeah, but i guess active bug supported are only recent versions, however wait till Timos answers > > > -- Best Regards MfG Robert Schetterer From csch at kiez.net Sat Sep 22 19:11:06 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 18:11:06 +0200 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 Message-ID: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Ok, here's a toughie: Out of a whim (and because of the bad weather) I today decided to upgrade my completely functioning 2.0.15 installation on my Mac OS X 10.6.7 system. NB: It's not a Mac OS X "Server", as sold by Apple - I have compiled my dovecots myself for quite a while. Anyhow: At first everything appeared to work fine after the upgrade. Up until I created a new user and -while testing- SELECTed his INBOX. The SELECT was "stuck". Though the process seemed to be alive I could only get rid of it with a "kill -9 ". The same happened when selecting a newly created mailbox on any of the other (otherwise functioning) accounts, so the problem had nothing to do with the new account, but rather with the fact, that it's INBOX was empty. Using dtruss and gdb I found out, that the dovecot process was trying to obtain a GUID and in the course of doing so invoked gethostbyname(), which intern caused a lot of "mach message" handling and somewhere, deep down there, the process was stuck. Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: sudo /usr/libexec/StartupItemContext dovecot (again: all this was working fine under 2.0.15) Now with 2.1.10, when I manually invoke dovecot with just sudo dovecot Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. Now -with all that said- here's my question: What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? It appears, that the imap process no longer "inherits" the StartupItemContext from the main process, so some change between 2.0.15 and 2.1.10 must have broken it ... Any help is highly appreciated - Clemens PS: I google'd around a lot and searched the mailing-lists, of course. I only found a post of someone who ran into the same/similar problem dating back to Feb 21st 2012 under the subject "dovecot freezes when trying to get mail from maildir with mail", but it was quickly dismissed without ever getting resolved and that was that. PS2: I intentionally didn't include any configs with this mail as they seem to be irrelevant, but of course I can generate the necessary output if needed. From tss at iki.fi Sat Sep 22 19:50:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:50:27 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <505C23E6.2070606@alec.pl> References: <505C23E6.2070606@alec.pl> Message-ID: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> On 21.9.2012, at 11.23, A.L.E.C wrote: > On 09/20/2012 06:01 PM, Timo Sirainen wrote: >> Thoughts? Any better name for the command than "multi"? > > How about 'execute' or 'exec'. v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. From tss at iki.fi Sat Sep 22 19:52:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 22 Sep 2012 19:52:39 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <87mx0jbhye.fsf@dod.no> References: <87mx0jbhye.fsf@dod.no> Message-ID: On 21.9.2012, at 16.53, Steinar Bang wrote: >>>>>> Timo Sirainen : > >> Thoughts? > > How about: > a) Dropping into a command prompt where it's possible to give commands? > b) Making it possible to feed commands through a pipe (ie reading lines > from stdin)? Both of these would require adding some kind of new command parameter parsing. Currently shell does all the "multi word string" parsing and escaping and such. My original thought was also to be able to read commands from stdin, but I'd rather not add such special command parser, at least not yet.. From jcblanco at fi.upm.es Sat Sep 22 20:29:46 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Sat, 22 Sep 2012 19:29:46 +0200 Subject: [Dovecot] Released Pigeonhole v0.3.2 for Dovecot v2.1.9 In-Reply-To: <505A09D0.6040607@rename-it.nl> References: <5058BAED.5080505@fi.upm.es> <5058C22F.1070704@rename-it.nl> <5059F145.1030803@fi.upm.es> <505A09D0.6040607@rename-it.nl> Message-ID: <505DF58A.3020403@fi.upm.es> Sorry I've sent a config.log but get retained for approval. This is the beginning of the file: This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by Pigeonhole configure 0.3.3, which was generated by GNU Autoconf 2.59. Invocation command line was $ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --target=x86_64-redhat-linux-gnu --program-prefix= --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --with-dovecot=/home/jc/rpmbuild/BUILD/dovecot-2.1.10 --with-managesieve=yes --enable-header-install=yes INSTALL_DATA=install -c -p -m644 ## --------- ## ## Platform. ## ## --------- ## hostname = testv.fi.upm.es uname -m = x86_64 uname -r = 2.6.18-308.13.1.el5debug uname -s = Linux uname -v = #1 SMP Tue Aug 21 18:34:48 EDT 2012 /usr/bin/uname -p = unknown /bin/uname -X = unknown /bin/arch = x86_64 /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown hostinfo = unknown /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown Regards Juan C. Blanco On 19/09/2012 20:07, Stephan Bosch wrote: > On 9/19/2012 6:22 PM, Juan C. Blanco wrote: >> On 18/09/2012 20:49, Stephan Bosch wrote: >>> On 9/18/2012 8:18 PM, Juan C. Blanco wrote: >>>> It seems that the error is related to the ld.gold fix, If I revert the >>>> c52a0c561311 patch the error is not produced >>> >>> Oh d'oh. Yes, it related to that. I fixed it for Dovecot v2.2 before, >>> but I forgot the Dovecot v2.1 tree. Fixed now: >>> >>> http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/66adbdd89d5c > > D'oh, this is the v2.2 change. The URL I meant is this one: > > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/8144fb698cb3 >>> >>> This only happens when you compile against installed Dovecot headers, >>> people using --with-dovecot= (like me) would not have seen this. >>> >>> I'll push out a new release once 2.1.10 comes out. I'm being told that >>> is imminent. >> >> Sorry, but using the new 0.3.3 version I'm getting the same error and >> I'm also using --with-dovecot=... with the dovecot 2.1.10 tree in >> configure, not the installed headers. If needed I can send you the >> configure invocation command. > > Yes, that could be helpful. > > Regards, > > Stephan. > From csch at kiez.net Sat Sep 22 22:51:22 2012 From: csch at kiez.net (Clemens Schrimpe) Date: Sat, 22 Sep 2012 21:51:22 +0200 Subject: [Dovecot] Update: Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 (and 2.0.21) References: <692EAFE7-E424-4F95-8DFA-C963B4957E11@kiez.net> Message-ID: Update: I tried with 2.0.21 and this also works just fine. So it must be something which came in with 2.1.x PS: Where is the documentaion for 2.1.x - i.e. for all the nice additions Timo made? The Website "only" has 2.0.x, as far as I can tell? (might be wrong here - hadn't touched the whole thing for a while, as my "civil life" had occupied me :-) Greetings, Clemens From joe at netmusician.org Sat Sep 22 22:52:42 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:52:42 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E170A.5090800@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sat Sep 22 22:57:03 2012 From: joe at netmusician.org (Joe Auty) Date: Sat, 22 Sep 2012 15:57:03 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 Message-ID: <505E180F.5060407@netmusician.org> Hello, I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 > sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] I'm using the 2.1.10 packages obtained with the following in my sources.list: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs. From mr88talent at gmail.com Sun Sep 23 04:00:21 2012 From: mr88talent at gmail.com (Gary V) Date: Sat, 22 Sep 2012 19:00:21 -0600 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On Sat, Sep 22, 2012 at 1:57 PM, Joe Auty wrote: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I can > try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp >> 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple SSL > certs. May not be related at all, but when I see segfaults on Debian, the first thing I try is to switch to bash as the default system shell: dpkg-reconfigure dash Answer: Use dash as the default system shell (/bin/sh)? -- Gary V From joe at netmusician.org Sun Sep 23 07:58:54 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 00:58:54 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505E970E.1040803@netmusician.org> > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: postbox-contact.jpg Type: image/jpeg Size: 1305 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nmtwitter.png Type: image/png Size: 1674 bytes Desc: not available URL: From joe at netmusician.org Sun Sep 23 08:03:28 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 01:03:28 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E970E.1040803@netmusician.org> References: <505E180F.5060407@netmusician.org> <505E970E.1040803@netmusician.org> Message-ID: <505E9820.5070007@netmusician.org> > > Gary V > September 22, 2012 9:00 PM > > May not be related at all, but when I see segfaults on Debian, the > first thing I try is to switch to bash as the default system shell: > > dpkg-reconfigure dash > > Answer: Use dash as the default system shell (/bin/sh)? Unfortunately this hasn't completely resolved the issue (I don't know if it has reduced the number of times this has happened either). FWIW I'm getting the same sort of segfault 28 with both imap-login and pop-login. I'm using dovecot-ldap, if that makes a difference. I'm not really sure what the best way would be to trace this problem so that I can provide you guys with more useful info. P.S. sorry for the duplicate message, it seems to have something to do with attaching images to messages. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ > From robert at schetterer.org Sun Sep 23 08:44:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 07:44:36 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: <505EA1C4.2000808@schetterer.org> Am 22.09.2012 21:57, schrieb Joe Auty: > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what I > can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 >> sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling multiple > SSL certs. show your dovecot conf -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 09:00:10 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 02:00:10 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA1C4.2000808@schetterer.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> Message-ID: <505EA56A.6010102@netmusician.org> > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf Thanks! http://pastebin.com/gVRrCjyK I took what you said literally, please let me know if you'd like to see any of the other Debian config files. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org From weber at zackbummfertig.de Sun Sep 23 10:56:52 2012 From: weber at zackbummfertig.de (weber at zackbummfertig.de) Date: Sun, 23 Sep 2012 09:56:52 +0200 Subject: [Dovecot] Dovecot Clustering with dsync over ssh Message-ID: Hello List, i only found an old post of tiemo where he announced that he will do an wiki article how to setup dovecot cluster with dsync over ssh. anyone knows if this artcile exists and where? i would like to setup a dovecot cluster with 2 machines. thanks for help marko From dovecot-list at mohtex.net Sun Sep 23 12:07:29 2012 From: dovecot-list at mohtex.net (Tamsy) Date: Sun, 23 Sep 2012 16:07:29 +0700 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505ED151.8090807@mohtex.net> Joe Auty wrote the following on 23.09.2012 13:00: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > With "show your dovecot conf" Robert means show us the output of "doveconf -n" From joe at netmusician.org Sun Sep 23 12:16:15 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 05:16:15 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED151.8090807@mohtex.net> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> Message-ID: <505ED35F.1020402@netmusician.org> > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe at netmusician.org -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: From tss at iki.fi Sun Sep 23 12:58:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 23 Sep 2012 12:58:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505E180F.5060407@netmusician.org> References: <505E180F.5060407@netmusician.org> Message-ID: On 22.9.2012, at 22.57, Joe Auty wrote: > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this? > >> [7236671.301179] imap-login[9306]: segfault at 28 ip 00007f9ff83c7e14 sp 00007fff49d385c0 error 4 in libdovecot.so.0.0.0[7f9ff837c000+8b000] You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work: service imap-login { executable = imap-login -D } Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full From robert at schetterer.org Sun Sep 23 13:27:15 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 23 Sep 2012 12:27:15 +0200 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505EE403.8060209@schetterer.org> Am 23.09.2012 08:00, schrieb Joe Auty: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK fine, but what i/we like to see is the result of dovecot -n on the list, and pleady avoid mail your jpg only to the list ( what ever this is ) > > I took what you said literally, please let me know if you'd like to see > any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I >> suspect this might be causing performance issues. Any suggestions what >> I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > -- Best Regards MfG Robert Schetterer From joe at netmusician.org Sun Sep 23 13:44:23 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 06:44:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505ED35F.1020402@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505ED151.8090807@mohtex.net> <505ED35F.1020402@netmusician.org> Message-ID: <505EE807.40903@netmusician.org> > > Tamsy > September 23, 2012 5:07 AM > > With "show your dovecot conf" Robert means show us the output of > "doveconf -n" My apologies! Here it is... # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_verbose = yes disable_plaintext_auth = no mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/doriandata/dovecotindexes/%u mail_nfs_storage = yes namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = " imap pop3" ssl_cert = > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From joe at netmusician.org Sun Sep 23 14:05:36 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 07:05:36 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> Message-ID: <505EED00.6090109@netmusician.org> > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full I hope I'm doing this correctly! # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...(no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f789cf5e018 in login_binary_run () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #5 0x0000000000402459 in ?? () No symbol table info available. #6 0x00007fff8a9c65f8 in ?? () No symbol table info available. #7 0x000000000000001c in ?? () No symbol table info available. #8 0x0000000000000002 in ?? () No symbol table info available. #9 0x00007fff8a9c7e6a in ?? () No symbol table info available. #10 0x00007fff8a9c7e7d in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available. > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Sun Sep 23 16:51:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 09:51:23 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EA56A.6010102@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> Message-ID: <505F13DB.1060400@Media-Brokers.com> On 2012-09-23 2:00 AM, Joe Auty wrote: >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK Please don't make it harder for others to help you... Just paste the output in the body of the email. Some (many?) people won't click on links like that from people they don't know... -- Best regards, Charles From stephan at rename-it.nl Sun Sep 23 20:31:27 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:31:27 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <20120920081126.0627cd71@tomh> References: <20120920081126.0627cd71@tomh> Message-ID: <505F476F.5090305@rename-it.nl> On 9/20/2012 2:11 PM, Tom Horsley wrote: > A slightly off topic question, but maybe someone here has seen > something like this: > > I want a program that operates entirely on the client side of > an IMAP server, and implements the sieve filtering language > by doing the necessary client side operations to move mail > on the IMAP server, expunge mail, etc. > > It would also act as a proxy between my real mail client and > the IMAP server so I'd never see any mail that hasn't been > filtered first. > > The benefits are obvious: I could finally get sieve filtering > on my stupid corporate exchange mail server :-). Interesting idea. I've been experimenting a bit with the sieve-filter command line tool and the Dovecot imapc mail storage backend. This does not entirely match your proposed solution, since it'll have to operate as a cron job, but at least it is a start. I've configured my experiment as follows: => I built a simple dovecot.conf protocols = none listen = *, :: mail_location = imapc:~/imapc imapc_host = your.host.tld imapc_port = 143 imapc_user = username imapc_password = imapc_ssl = starttls imapc_ssl_verify = no => I tested whether IMAP connection works: stephan at klara:~/test$ /usr/lib/dovecot/imap -c ./dovecot.conf * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in as stephan 2342 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8274 EXISTS * 0 RECENT * OK [UIDVALIDITY 1348169448] UIDs valid * OK [UIDNEXT 8275] Predicted next UID * OK [NOMODSEQ] No permanent modsequences 2342 OK [READ-WRITE] Select completed. => I built a Sieve script called test.sieve: require "fileinto"; require "mailbox"; # Just moves mail elsewhere fileinto :create "Target"; => And finally tested operation using sieve-filter: sieve-filter -v -e -W -c dovecot.conf test.sieve INBOX In the process a few problems were encountered and solved. This means that this little experiment will only work with the latest revisions of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective Mercurial repositories. The conclusion so far is that, in essence, what you'd like to do is possible. However, to make Sieve filtering look like it is being run at delivery rather than at regular intervals in the background, we will need to build some sort of IMAP plugin that monitors INBOX for new messages and passes them through a Sieve filter before the user's proxied IMAP client can see them. I'll have to investigate how much work implementing this would be. Regards, Stephan. From stephan at rename-it.nl Sun Sep 23 20:43:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 19:43:41 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <505F476F.5090305@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> Message-ID: <505F4A4D.6060304@rename-it.nl> On 9/23/2012 7:31 PM, Stephan Bosch wrote: > On 9/20/2012 2:11 PM, Tom Horsley wrote: >> I want a program that operates entirely on the client side of >> an IMAP server, and implements the sieve filtering language >> by doing the necessary client side operations to move mail >> on the IMAP server, expunge mail, etc. >> > Interesting idea. I've been experimenting a bit with the sieve-filter > command line tool and the Dovecot imapc mail storage backend. This > does not entirely match your proposed solution, since it'll have to > operate as a cron job, but at least it is a start. [...] > In the process a few problems were encountered and solved. This means > that this little experiment will only work with the latest revisions > of Dovecot v2.1/v2.2 and a matching Pigeonhole from their respective > Mercurial repositories. > > The conclusion so far is that, in essence, what you'd like to do is > possible. However, to make Sieve filtering look like it is being run > at delivery rather than at regular intervals in the background, we > will need to build some sort of IMAP plugin that monitors INBOX for > new messages and passes them through a Sieve filter before the user's > proxied IMAP client can see them. I'll have to investigate how much > work implementing this would be. I should explain explain in a little more detail how this works. Basically, I use the sieve-filter tool to filter mail that is already in a mailbox. Only, the mailbox is a bit special: it is an imapc (IMAP Client) mailbox that proxies to a remote mailbox. So, on our local machine the INBOX actually maps to INBOX on the remote server, rather than some local store. For the sieve-filter tool this makes no difference. This way, the sieve-filter can also be used to filter mail on a remote server. Regards, Stephan. From me at junc.org Sun Sep 23 20:47:17 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 23 Sep 2012 19:47:17 +0200 Subject: [Dovecot] =?utf-8?q?Sieve_as_mail_client=3F?= In-Reply-To: <505F4A4D.6060304@rename-it.nl> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> Message-ID: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Stephan Bosch skrev den 23-09-2012 19:43: > This way, the sieve-filter can also be used to filter mail > on a remote server. is it gpl ?, anyway pretty cool From stephan at rename-it.nl Sun Sep 23 21:44:36 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 23 Sep 2012 20:44:36 +0200 Subject: [Dovecot] Sieve as mail client? In-Reply-To: <2fe063d4e1ea2f5361fe2cba87717797@junc.org> References: <20120920081126.0627cd71@tomh> <505F476F.5090305@rename-it.nl> <505F4A4D.6060304@rename-it.nl> <2fe063d4e1ea2f5361fe2cba87717797@junc.org> Message-ID: <505F5894.8040809@rename-it.nl> On 9/23/2012 7:47 PM, Benny Pedersen wrote: > Stephan Bosch skrev den 23-09-2012 19:43: >> This way, the sieve-filter can also be used to filter mail >> on a remote server. > > is it gpl ? The sieve-filter tool is part of Pigeonhole. Both Dovecot and Pigeonhole are mostly LGPLv2.1. Check the COPYING files for details. Regards, Stephan. From joe at netmusician.org Sun Sep 23 22:45:33 2012 From: joe at netmusician.org (Joe Auty) Date: Sun, 23 Sep 2012 15:45:33 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F13DB.1060400@Media-Brokers.com> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> Message-ID: <505F66DD.70603@netmusician.org> > > Charles Marcus > September 23, 2012 9:51 AM > > > Please don't make it harder for others to help you... > > Just paste the output in the body of the email. Some (many?) people > won't click on links like that from people they don't know... My apologies, I figured at the time that it would be better to not make these emails crazy long, but I guess they can be just as easily snipped... Here is my conf: ## Dovecot configuration file # If you're in a hurry, see http://wiki2.dovecot.org/QuickConfiguration # "doveconf -n" command gives a clean output of the changed settings. Use it # instead of copy&pasting files when posting to the Dovecot mailing list. # '#' character and everything after it is treated as comments. Extra spaces # and tabs are ignored. If you want to use either of these explicitly, put the # value inside quotes, eg.: key = "# char and trailing whitespace " # Default values are shown for each setting, it's not required to uncomment # those. These are exceptions to this though: No sections (e.g. namespace {}) # or plugin settings are added by default, they're listed only as examples. # Paths are also just examples with the real defaults being based on configure # options. The paths listed here are for configure --prefix=/usr # --sysconfdir=/etc --localstatedir=/var # Enable installed protocols !include_try /usr/share/dovecot/protocols.d/*.protocol # A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, :: # Base directory where to store runtime data. #base_dir = /var/run/dovecot/ # Name of this instance. In multi-instance setup doveadm and other commands # can use -i to select which instance is used (an alternative # to -c ). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot # Greeting message for clients. #login_greeting = Dovecot ready. # Space separated list of trusted network ranges. Connections from these # IPs are allowed to override their IP addresses and ports (for logging and # for authentication checks). disable_plaintext_auth is also ignored for # these networks. Typically you'd specify your IMAP proxy servers here. #login_trusted_networks = # Sepace separated list of login access check sockets (e.g. tcpwrap) #login_access_sockets = # With proxy_maybe=yes if proxy destination matches any of these IPs, don't do # proxying. This isn't necessary normally, but may be useful if the destination # IP is e.g. a load balancer's IP. #auth_proxy_self = # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). #verbose_proctitle = no verbose_proctitle = yes # Should all processes be killed when Dovecot master process shuts down. # Setting this to "no" means that Dovecot can be upgraded without # forcing existing client connections to close (although that could also be # a problem if the upgrade is e.g. because of a security fix). #shutdown_clients = yes # If non-zero, run mail commands via this many connections to doveadm server, # instead of running them directly in the same process. #doveadm_worker_count = 0 # UNIX socket or host:port used for connecting to doveadm server #doveadm_socket_path = doveadm-server # Space separated list of environment variables that are preserved on Dovecot # startup and passed down to all of its child processes. You can also give # key=value pairs to always set specific settings. #import_environment = TZ ## ## Dictionary server settings ## # Dictionary can be used to store key=value lists. This is used by several # plugins. The dictionary can be accessed either directly or though a # dictionary server. The following dict block maps dictionary names to URIs # when the server is used. These can then be referenced using URIs in format # "proxy::". dict { #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext } # Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf # A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf > > > Joe Auty > September 23, 2012 2:00 AM >> >> Robert Schetterer >> September 23, 2012 1:44 AM >> >> show your dovecot conf > > Thanks! > > http://pastebin.com/gVRrCjyK > > I took what you said literally, please let me know if you'd like to > see any of the other Debian config files. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > Robert Schetterer > September 23, 2012 1:44 AM > > show your dovecot conf > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From CMarcus at Media-Brokers.com Mon Sep 24 01:42:09 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 23 Sep 2012 18:42:09 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505F66DD.70603@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EA1C4.2000808@schetterer.org> <505EA56A.6010102@netmusician.org> <505F13DB.1060400@Media-Brokers.com> <505F66DD.70603@netmusician.org> Message-ID: <505F9041.6080700@Media-Brokers.com> On 2012-09-23 3:45 PM, Joe Auty wrote: >> >> Charles Marcus >> September 23, 2012 9:51 AM >> >> >> Please don't make it harder for others to help you... >> >> Just paste the output in the body of the email. Some (many?) people >> won't click on links like that from people they don't know... > > My apologies, I figured at the time that it would be better to not > make these emails crazy long, but I guess they can be just as easily > snipped... Here is my conf: > > ## Dovecot configuration file No, you got it right the second time... When I said 'output', I mean the output of doveconf -n... which you did in a follow up email... -- Best regards, Charles From amateo at um.es Mon Sep 24 09:17:11 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 24 Sep 2012 08:17:11 +0200 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505C3435.30601@blueboard.cz> References: <505C3435.30601@blueboard.cz> Message-ID: <505FFAE7.3070905@um.es> El 21/09/12 11:32, Tom?? Randa escribi?: > Hello, > > I still cannot get dovecot running with more then 1000 processes, but > hard limit is 8192 per user in box. I tried everything, including > modifying startup script of dovecot to set ulimit -u 8192. Could it be > some dovecot bug or dovecot<>freebsd bug? > I also tried to set client_limit=2 in imap service to spawn more imap > clients in one process, but still I am over 1000 processes with kernel > message: > > maxproc limit exceeded by uid 89 > > > Could anybody help? Many thanks Tomas > Hi, I don't know BSD, but we had a similar problems with linux, when we reached 1024 processes, no more processes were created and we had errors like "imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument". If this is your same case, you could look for more info at http://www.dovecot.org/list/dovecot/2012-July/067014.html -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From manu at netbsd.org Mon Sep 24 10:39:54 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Mon, 24 Sep 2012 07:39:54 +0000 Subject: [Dovecot] dsync Message-ID: <20120924073954.GB16335@homeworld.netbsd.org> Hi Testing dsync, things go wrong: doveadm sync -u user remote:root at mail2.example.net dsync-local(user): Error: Mailboxes don't have unique GUIDs: 72e3be2c6f203b50883c000044af56a8 is shared by RT and RT_72e3be2c6f203b50883c000044af56a8 Obviously RT_72e3be2c6f203b50883c000044af56a8 is an outdated copy of RT But .mailboxlist does not list that mailbox. Is there a trick to make sure dsync only use valid mailboxes? I have this in dovecot.conf mail_location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=/mail/indexes/%u:SUBSCRIPTI ONS=../.mailboxlist Another problem, that may or may not be related: dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 60298748 dsync-local(user): Error: Failed to sync mailbox RT: Timeout while waiting for lock dsync-local(user): Error: Next message unexpectedly corrupted in mbox file /home/user/mail/RT at 63587421 dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files I also get this: dsync-local(user): Error: Failed to sync mailbox RT: Mailbox GUIDs are not permanent without index files dsync-local(user): Error: proxy client timed out (waiting for MSG-GET message from remote) And this: dsync-local(user): Error: read() from worker server failed: EOF And generally speaking ,how good is dsync? is it usabel in production? This is on dovecot 2.1.7 -- Emmanuel Dreyfus manu at netbsd.org From dovecot at lists.wgwh.ch Mon Sep 24 11:49:22 2012 From: dovecot at lists.wgwh.ch (Oli Schacher) Date: Mon, 24 Sep 2012 10:49:22 +0200 Subject: [Dovecot] 2.1.10 imapc assert crash report Message-ID: <20120924104922.211d9ca3@boscos> Hi Timo I have a simple imapc gmail proxy test setup which works fine on 2.1.9, but crashes on 2.1.10 # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.4.7-1-ARCH x86_64 auth_mechanisms = plain login imapc_host = imap.gmail.com imapc_port = 993 imapc_ssl = imaps imapc_ssl_ca_dir = /etc/ssl/certs listen = 127.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=imap.gmail.com port=993 ssl=imaps default_fields = userdb_imapc_user=%u userdb_imapc_password=%w userdb_imapc_ssl=imaps userdb_imapc_port=993 driver = imap } protocols = imap ssl = no userdb { driver = prefetch } Log: Sep 24 10:21:58 codemonkey dovecot: master: Dovecot v2.1.10 starting up Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Sep 24 10:22:12 codemonkey dovecot: auth: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x453aa) [0x7f8d5ce963aa] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x453ee) [0x7f8d5ce963ee] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8d5ce6abd3] -> /usr/local/lib/dovecot/auth/libauthdb_imap.so(+0x977c) [0x7f8d5be3677c] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3618e) [0x7f8d5ce8718e] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f8d5cea3006] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7f8d5cea3df7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f8d5cea2b48] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8d5ce8eb93] -> dovecot/auth(main+0x2ff) [0x40ad4f] -> /lib/libc.so.6(__libc_start_main+0xf5) [0x7f8d5c27d725] -> dovecot/auth() [0x40af61] Sep 24 10:22:12 codemonkey dovecot: auth: Fatal: master: service(auth): child 24008 killed with signal 6 (core not dumped) Sep 24 10:22:12 codemonkey dovecot: imap-login: Warning: Auth connection closed with 1 pending requests (max 0 secs, pid=24007, EOF) Oli -- message transmitted on 100% recycled electrons From tss at iki.fi Mon Sep 24 14:11:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:11:24 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: On 21.9.2012, at 8.28, Timo Sirainen wrote: >> Timo Sirainen wrote: >>> doveadm multi [-A | -u wildcards] [ [...]] >>> >>> Thoughts? > >> As command name I could also think of "doveadm sequence", which >> implies the commands being executed in serial order. > > Hmm. Maybe. "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. From tss at iki.fi Mon Sep 24 14:17:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:17:16 +0300 Subject: [Dovecot] 2.1.10 imapc assert crash report In-Reply-To: <20120924104922.211d9ca3@boscos> References: <20120924104922.211d9ca3@boscos> Message-ID: <0734901B-7805-42C5-9039-F0D6C2EC566B@iki.fi> On 24.9.2012, at 11.49, Oli Schacher wrote: > I have a simple imapc gmail proxy test setup which works fine on 2.1.9, > but crashes on 2.1.10 > Sep 24 10:22:12 codemonkey dovecot: auth: Panic: file imapc-connection.c: line 1289 (imapc_connection_connect_next_ip): assertion failed: (conn->client->set.max_idle_time > 0) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/fd863826c892 http://hg.dovecot.org/dovecot-2.1/rev/17a8f15beb8c From Ralf.Hildebrandt at charite.de Mon Sep 24 14:27:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:27:06 +0200 Subject: [Dovecot] Logging question regarding delete actions Message-ID: <20120924112706.GS4043@charite.de> A user is logged in via imap from multiple devices. The log has this: Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 How can I find out WHICH CLIENT caused the deletion? Same issue: Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 How can I be sure that the log entry from 10:06:17 "belongs" to the log entry from "09:36:05"? Also, what is the meaning of the "session="? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:36:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:36:45 +0300 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <20120924112706.GS4043@charite.de> References: <20120924112706.GS4043@charite.de> Message-ID: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> On 24.9.2012, at 14.27, Ralf Hildebrandt wrote: > A user is logged in via imap from multiple devices. > The log has this: > > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15347, msgid=<1341851741.4ffb085d2e2b7 at swift.generated>, size=15675 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15739, msgid=, size=18134 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15740, msgid=<3509D8B3F9054C4BB26C85F3E4B96563036822877538 at EXCHANGE21.charite.de>, size=22226 > Sep 21 11:46:32 postamt dovecot: imap(awxxxxer): delete: box=INBOX, uid=15741, msgid=, size=5160 > > How can I find out WHICH CLIENT caused the deletion? Change mail_log_prefix to include %{session} (and maybe %r for IP). > Same issue: > > Sep 21 09:36:05 postamt dovecot: imap-login: Login: user=, method=PLAIN, rip=109.45.0.37, lip=141.42.206.36, mpid=30773, TLS, session= > Sep 21 10:06:17 postamt dovecot: imap(awxxxxer): Disconnected for inactivity in=2255 out=4398 > > How can I be sure that the log entry from 10:06:17 "belongs" to the > log entry from "09:36:05"? Also, what is the meaning of the > "session="? This is also solved with mail_log_prefix change. The session's idea is exactly to match the same session's log messages together. It's a string guaranteed to be unique for the next .. was it 7 years or so. From alec at alec.pl Mon Sep 24 14:44:29 2012 From: alec at alec.pl (A.L.E.C) Date: Mon, 24 Sep 2012 13:44:29 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> Message-ID: <5060479D.3060609@alec.pl> On 09/22/2012 06:50 PM, Timo Sirainen wrote: > On 21.9.2012, at 11.23, A.L.E.C wrote: > >> On 09/20/2012 06:01 PM, Timo Sirainen wrote: >>> Thoughts? Any better name for the command than "multi"? >> >> How about 'execute' or 'exec'. > > v2.1.10 already has "dovecot exec" that does a different thing. So can't be anything related to "exec".. next is "run" or "pipe", but what if you create global separator option and detect multi-command syntax usage automatically without a keyword? Syntax for doveadm would be doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command [command_options] [command_arguments] [separator command [command_options] [command_arguments] [...]] and example doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl From Ralf.Hildebrandt at charite.de Mon Sep 24 14:56:45 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 24 Sep 2012 13:56:45 +0200 Subject: [Dovecot] Logging question regarding delete actions In-Reply-To: <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> References: <20120924112706.GS4043@charite.de> <91CFB975-55A4-4E2B-9407-A7CD675671FA@iki.fi> Message-ID: <20120924115645.GY4043@charite.de> * Timo Sirainen : > This is also solved with mail_log_prefix change. The session's idea is > exactly to match the same session's log messages together. It's a > string guaranteed to be unique for the next .. was it 7 years or so. Thanks. I changed the mail_log_prefix from mail_log_prefix = "%s(%u): " to mail_log_prefix = "%s(%u) %{session}: " -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Mon Sep 24 14:58:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 14:58:53 +0300 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <5060479D.3060609@alec.pl> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> Message-ID: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> On 24.9.2012, at 14.44, A.L.E.C wrote: > next is "run" or "pipe", but what if you create global separator option > and detect multi-command syntax usage automatically without a keyword? > > Syntax for doveadm would be > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > [command_options] [command_arguments] [separator command > [command_options] [command_arguments] [...]] > > and example > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge Hmm. Yes, that might work. Although it would have to be: doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge because both -A and -s are mail command specific parameters, which won't work for non-mail commands. Hmm. This reminds me also that it would be possible with some extra work to do some command interaction. IMAP supports saving search results, which can later be accessed with $ parameter. So this could be made to work: doveadm search -s : from foo : fetch text \$ : expunge \$ From chevalier at dspnet.fr Mon Sep 24 16:48:11 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 15:48:11 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) Message-ID: <20120924134810.GA62723@kyoko.org> Hello, I don't know if it's been addressed before, but anyway : In my dovecot setup, I have local and virtual users. So, I need multiple passdb backends. Namely, passwd for the local users and ldap for the virtual users. passdb { driver = passwd } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } Everything work correctly : when a user logs in (imap/pop3) there's a lookup in passwd and if it fails there's a lookup in ldap (if I understand the process correctly), which eventually succeeds. Except that every time a virtual user logs in, dovecot logs an error, like : dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument I guess it's because the login is a full email that getpwnam fails. Anyway, the user logs in just fine. But I would like to know if/how I can get rid of the messages filling my logs ? I tried : auth_debug_passwords = no auth_verbose = no But no dice. I used dovecot 1.x before and there was no such messages. Thanks for any advice. K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From tss at iki.fi Mon Sep 24 17:16:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:16:06 +0300 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924134810.GA62723@kyoko.org> References: <20120924134810.GA62723@kyoko.org> Message-ID: On 24.9.2012, at 16.48, Philippe Chevalier wrote: > dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument > > I guess it's because the login is a full email that getpwnam fails. So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 386 bytes Desc: not available URL: From tss at iki.fi Mon Sep 24 17:32:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:32:22 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <505EED00.6090109@netmusician.org> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: On 23.9.2012, at 14.05, Joe Auty wrote: > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? From blevi.linux at gmail.com Mon Sep 24 17:55:18 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 17:55:18 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <50607456.1040709@gmail.com> On 24/09/2012 17:32, Timo Sirainen wrote: > On 23.9.2012, at 14.05, Joe Auty wrote: > >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> (gdb) bt full >> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 > > Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Levi From tss at iki.fi Mon Sep 24 17:58:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 24 Sep 2012 17:58:17 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <50607456.1040709@gmail.com> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> Message-ID: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> On 24.9.2012, at 17.55, Birta Levente wrote: > On 24/09/2012 17:32, Timo Sirainen wrote: >> On 23.9.2012, at 14.05, Joe Auty wrote: >> >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> (gdb) bt full >>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> No symbol table info available. >>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >> >> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >> > > I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? From blevi.linux at gmail.com Mon Sep 24 18:04:29 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Mon, 24 Sep 2012 18:04:29 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <5060767D.50002@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > I build my own rpm based on src rpm: dovecot-2.1.1-2_132.src.rpm. #dovecot -n auth_mechanisms = plain login cram-md5 debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no listen = * mail_access_groups = vmail mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = quota mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Quota exceeded, please contact postmaster at benvenuti.ro quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=85%% quota-warning 85 %u quota_warning4 = storage=80%% quota-warning 80 %u quota_warning5 = storage=50%% quota-warning 50 %u } postmaster_address = postmaster at mydomain.com service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0640 user = vmail } user = vmail } ssl_ca = References: <20120924134810.GA62723@kyoko.org> Message-ID: <20120924150440.GA85969@kyoko.org> On Mon, Sep 24, 2012 at 05:16:06PM +0300, Timo Sirainen wrote: >On 24.9.2012, at 16.48, Philippe Chevalier wrote: > >> dovecot: auth-worker(99126): Error: passwd(xxx at domain.org,12.34.254.255): getpwnam() failed: Invalid argument >> >> I guess it's because the login is a full email that getpwnam fails. > >So if you log in as nonexistent user "foo.bar" it doesn't log an error, but if you log in as "foo at bar" it does? The attached patch probably fixes it? If I log in as a non-existent user (neither in passwd or ldap), without the domain part, it logs also an error, but this time from ldap : dovecot: auth: Error: ldap(foo.bar,xx.xx.xx.xx,): ldap_bind() failed: Invalid DN syntax My bind DN to check the password is : auth_bind_userdn = dc=%n,dc=%d,ou=Domains,ou=Mail,dc=dspnet,dc=fr (I have virtual users in multiple domains) So ldap protests probably because the "domain" part is missing. If I use a non-existent login "foo at bar", dovecot logs nothing : no error from passwd, no error from ldap, just an authentication error on the client side. I will apply the patch later today and will let you know the result. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From alessio at skye.it Mon Sep 24 18:40:52 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 17:40:52 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <5059C393.5050209@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> Message-ID: <50607F04.9040104@skye.it> Il 19/09/2012 15:07, Alessio Cecchi ha scritto: > Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>> >>>> LDA is configured and works fine but the problem is when the first >>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>> message is written to the user's Mailbox but the message remains, >>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>> the second attempt is delivered fine. >>> gdb backtrace would be very helpful in figuring out the problem: >>> http://dovecot.org/bugreport.html >>> >> Hi Timo, had you occasion to see the problem? Can I provide more information? Thanks > > This is the full bt: > > (gdb) bt full > #0 acl_lookup_dict_rebuild (dict=0x0) at acl-lookup-dict.c:221 > ns = > ids_arr = {arr = {buffer = 0x0, element_size = 26492496}, v = > 0x0, > v_modifiable = 0x0} > ids = 0x1928658 > i = > dest = > ret = -883075307 > #1 0x00007f2fc9fc41b4 in acl_backend_vfile_acllist_try_rebuild ( > backend=0x1944240) at acl-backend-vfile-acllist.c:297 > auser = 0x1949a08 > iter = 0x0 > acllist_path = 0x1928658 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir/dovecot-acl-list" > ret = > ns = 0x1943e50 > output = 0x0 > st = {st_dev = 2051, st_ino = 662103, st_nlink = 1, st_mode = > 33152, > st_uid = 89, st_gid = 89, __pad0 = 0, st_rdev = 0, st_size = 0, > st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = > 1348059559, > tv_nsec = 0}, st_mtim = {tv_sec = 1348059559, tv_nsec = 0}, > st_ctim = {tv_sec = 1348059559, tv_nsec = 0}, __unused = {0, > 0, 0}} > path = 0x1928210 > > file_mode = 384 > dir_mode = 448 > gid = 4294967295 > list = > info = > rootdir = 0x1928610 "Sent" > origin = 0x194d178 > "/home/vpopmail/domains/qboxdns.it/cecchi10/Maildir" > fd = 8 > #2 acl_backend_vfile_acllist_rebuild (backend=0x1944240) > at acl-backend-vfile-acllist.c:311 > acllist_path = > #3 0x00007f2fc9fc4563 in acl_backend_vfile_acllist_refresh > (backend=0x1944240) > at acl-backend-vfile-acllist.c:153 > __FUNCTION__ = "acl_backend_vfile_acllist_refresh" > #4 0x00007f2fc9fc46d5 in acl_backend_vfile_acllist_verify (backend=0x0, > name=0x1944a60 "", mtime=0) at acl-backend-vfile-acllist.c:343 > acllist = > #5 0x00007f2fc9fc30b8 in acl_backend_vfile_object_refresh_cache ( > _aclobj=0x19444e0) at acl-backend-vfile.c:858 > old_validity = > validity = {global_validity = {last_check = 0, > last_read_time = 1348059559, last_mtime = 0, last_size = 0}, > local_validity = {last_check = 0, last_read_time = 0, > > last_mtime = 0, last_size = 0}, mailbox_validity = { > last_check = 0, last_read_time = 0, last_mtime = 0, > last_size = 0}} > mtime = 0 > ret = 26515976 > #6 0x00007f2fc9fc125e in acl_backend_get_default_rights > (backend=0x1944240, > mask_r=0x28) at acl-backend.c:164 > No locals. > #7 0x00007f2fc9fc75bd in acl_mailbox_try_list_fast (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at acl-mailbox-list.c:107 > alist = > nonowner_list_ctx = > ret = > backend = 0x1944240 > acl_mask = 0x1 > ns = 0x1943e50 > update_ctx = {iter_ctx = 0x7f2fcb80d2c8, tree_ctx = > 0x7f2fcbf2ba88, > glob = 0x0, leaf_flags = 4294967295, parent_flags = 0, > update_only = 0, match_parents = 0} > name = > #8 acl_mailbox_list_iter_init (list=0x194cc00, patterns=0x7fff362dff50, > flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) at acl-mailbox-list.c:194 > _data_stack_cur_id = 2 > > ctx = 0x1946b20 > pool = > i = > inboxcase = > #9 0x00007f2fcb886d33 in mailbox_list_iter_init_multiple > (list=0x194cc00, > patterns=0x7fff362dff50, flags=MAILBOX_LIST_ITER_RETURN_NO_FLAGS) > at mailbox-list-iter.c:158 > ctx = > ret = > __FUNCTION__ = "mailbox_list_iter_init_multiple" > #10 0x00007f2fcb887459 in mailbox_list_iter_init (list=0x0, > pattern=, flags=1348059559) at > mailbox-list-iter.c:58 > patterns = {0x7f2fc9db76dc "*", 0x0} > #11 0x00007f2fc9db2370 in quota_count_namespace (root=0x1944950, > bytes_r=, count_r=0x7fff362dfff0) at > quota-count.c:73 > ctx = 0x7f2fcb5beef3 > info = > #12 quota_count (root=0x1944950, bytes_r=, > count_r=0x7fff362dfff0) at quota-count.c:111 > i = 0 > ret = 0 > #13 0x00007f2fc9db37ce in dict_quota_count (root=0x0, want_bytes=true, > value_r=0x7fff362e0038) at quota-dict.c:113 > > dt = > bytes = 0 > count = 0 > #14 0x00007f2fc9db395a in dict_quota_update_callback ( > ret=, context=0x1949a08) at quota-dict.c:178 > value = 1 > #15 0x00007f2fcb592258 in client_dict_finish_transaction (dict=0x194c7f0, > line_r=) at dict-client.c:265 > ctx = 0x1984190 > #16 client_dict_read_one_line (dict=0x194c7f0, line_r= out>) > at dict-client.c:356 > id = 1 > line = > ret = 0 > __FUNCTION__ = "client_dict_read_one_line" > #17 0x00007f2fcb592565 in client_dict_wait (_dict=) > at dict-client.c:520 > dict = 0x194c7f0 > line = 0x0 > ret = > #18 0x00007f2fc9db3ab5 in dict_quota_deinit (_root=) > at quota-dict.c:90 > root = 0x1944950 > > #19 0x00007f2fc9dafc72 in quota_root_deinit (root=0x0) at quota.c:240 > pool = 0x194c5c0 > #20 0x00007f2fc9db16e1 in quota_deinit (_quota=0x1949a38) at quota.c:335 > quota = 0x1944720 > i = 2 > #21 0x00007f2fc9db61dd in quota_user_deinit (user=0x19483c0) > at quota-storage.c:412 > quser = 0x1949a30 > quota_set = 0x1946f30 > #22 0x00007f2fcb88188e in mail_user_unref (_user=) > at mail-user.c:153 > user = 0x19483c0 > __FUNCTION__ = "mail_user_unref" > #23 0x0000000000402de2 in main (argc=3, argv=0x192d370) at main.c:481 > set_roots = {0x604640, 0x0} > ctx = {pool = 0x192df60, set = 0x1930720, session = 0x192df80, > dup_ctx = 0x0, session_id = 0x0, src_mail = 0x0, > src_envelope_sender = 0x0, dest_user = 0x0, > dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > final_dest_addr = 0x192d3c2 "cecchi10 at qboxdns.it", > dest_mailbox_name = 0x4034d9 "INBOX", dest_mail = 0x0, > var_expand_table = 0x192e050, tried_default_save = true, > saved_mail = true, save_dest_mail = false, mailbox_full = > false, > > dsn = false} > service_flags = > user = 0x192d3c2 "cecchi10 at qboxdns.it" > errstr = 0x0 > path = 0x7fff362e0368 "\351\a" > storage_service = 0x192f3a0 > service_user = 0x192fd48 > service_input = {module = 0x4034d5 "lda", service = 0x4034d5 > "lda", > username = 0x192d3c2 "cecchi10 at qboxdns.it", session_id = 0x0, > local_ip = {family = 0, u = {ip6 = {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = > {__in6_u = { > __u6_addr8 = '\000' , __u6_addr16 > = {0, 0, > 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, > ip4 = { > s_addr = 0}}}, local_port = 0, remote_port = 0, > userdb_fields = 0x0, flags_override_add = 0, > flags_override_remove = 0, no_userdb_lookup = 0} > storage = 0x1943f30 > user_source = > destaddr_source = 0x403594 "user at hostname" > process_euid = > stderr_rejection = false > > ret = > c = > error = MAIL_ERROR_NONE > (gdb) > > Sorry > -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From feltrin at gmail.com Mon Sep 24 18:44:52 2012 From: feltrin at gmail.com (Jean Michel) Date: Mon, 24 Sep 2012 12:44:52 -0300 Subject: [Dovecot] Patches and dovecot releases Message-ID: I'd like to know if after a release, for example the recently release 2.1.10, its common to see few days from the release some bug reports and algo some patches, are theses patches applyed on the daily builds ? ------------------------------ Jean Michel Feltrin From m.naumann at globalview.de Mon Sep 24 19:32:43 2012 From: m.naumann at globalview.de (M. Naumann) Date: Mon, 24 Sep 2012 18:32:43 +0200 Subject: [Dovecot] Traffic Accounting Message-ID: <50608B2B.5050207@globalview.de> Hi, I'm trying to find out how to do traffic accounting with Dovecot 2.x, preferrably v2.0.9, preferrably on CentOS 6. I've previously asked on IRC, but there was little feedback, and my understanding is now this list is the preferred media for such inquiries. If I recall correctly, some weeks ago I was told that traffic accounting is not officially supported on Dovecot 2, but that there could still be ways to get it to work, but no details were provided. I can think of the following approaches: * rawlog, preferrably piped (if that's possible?) into something like wc to prevent privacy issues and to reduce the I/O overhead * maildrop filtering in front of dovecot LDA (for mail inbound to mail storage) * sieve filtering Unfortunately I have little experience with either so far, so it's hard to make a good choice. I would appreciate hints on these approaches, and on any other approaches you can think of, as well as any related documentation / how-to you could point me to. While I'm subscribed to the list (for mail authentication purposes), I've disabled receiving any e-mail form the list, so please CC me on any replies. Thanks in advance, Moritz From robert at schetterer.org Mon Sep 24 19:46:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:46:23 +0200 Subject: [Dovecot] Patches and dovecot releases In-Reply-To: References: Message-ID: <50608E5F.2080704@schetterer.org> Am 24.09.2012 17:44, schrieb Jean Michel: > I'd like to know if after a release, for example the recently release > 2.1.10, its common to see few days from the release some bug reports and > algo some patches, are theses patches applyed on the daily builds ? > > > ------------------------------ > Jean Michel Feltrin > look http://hg.dovecot.org/ dovecot has tons of config options, and there are extrem multi ways to set it up ( for this we love dovecot ), so some stuff may bug in a stable release for some setups/functions sometimes also distros do self patching, there are sometimes kernel bugs etc in general its not a software having bugs, its the way and speed they get fixed, and Timo does a "wonderman" job here after all ,unless there are no security fixes, or you urgent need new setup options etc, nobody presses you to upgrade in a stable tree read the changelogs -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Sep 24 19:48:38 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 18:48:38 +0200 Subject: [Dovecot] Traffic Accounting In-Reply-To: <50608B2B.5050207@globalview.de> References: <50608B2B.5050207@globalview.de> Message-ID: <50608EE6.6030905@schetterer.org> Am 24.09.2012 18:32, schrieb M. Naumann: > Hi, > > I'm trying to find out how to do traffic accounting with Dovecot 2.x, > preferrably v2.0.9, preferrably on CentOS 6. > > I've previously asked on IRC, but there was little feedback, and my > understanding is now this list is the preferred media for such > inquiries. If I recall correctly, some weeks ago I was told that traffic > accounting is not officially supported on Dovecot 2, but that there > could still be ways to get it to work, but no details were provided. > > I can think of the following approaches: > > * rawlog, preferrably piped (if that's possible?) into something like wc > to prevent privacy issues and to reduce the I/O overhead > > * maildrop filtering in front of dovecot LDA (for mail inbound to mail > storage) > > * sieve filtering > > Unfortunately I have little experience with either so far, so it's hard > to make a good choice. I would appreciate hints on these approaches, and > on any other approaches you can think of, as well as any related > documentation / how-to you could point me to. > > While I'm subscribed to the list (for mail authentication purposes), > I've disabled receiving any e-mail form the list, so please CC me on any > replies. > > Thanks in advance, > > Moritz > perhaps this helps on the 2.1.x tree there is a stats plugin http://wiki2.dovecot.org/Statistics and/or use syslog analysers etc -- Best Regards MfG Robert Schetterer From ben at morrow.me.uk Mon Sep 24 20:01:02 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:01:02 +0100 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> References: <505C23E6.2070606@alec.pl> <77505A5B-F4C4-41ED-B1EA-0B8FCC79F37F@iki.fi> <5060479D.3060609@alec.pl> <8AA80CA6-CB55-4021-8140-993E171D04CC@iki.fi> Message-ID: <20120924170101.GB63656@anubis.morrow.me.uk> At 2PM +0300 on 24/09/12 you (Timo Sirainen) wrote: > On 24.9.2012, at 14.44, A.L.E.C wrote: > > > next is "run" or "pipe", but what if you create global separator option > > and detect multi-command syntax usage automatically without a keyword? > > > > Syntax for doveadm would be > > > > doveadm [-Dv] [-f formatter] [-s separator] [-A | -u wildcards ] command > > [command_options] [command_arguments] [separator command > > [command_options] [command_arguments] [...]] > > > > and example > > > > doveadm -A -s : expunge mailbox Trash savedbefore 7d : purge > > Hmm. Yes, that might work. Although it would have to be: > > doveadm expunge -A -s : mailbox Trash savedbefore 7d : purge > > because both -A and -s are mail command specific parameters, which > won't work for non-mail commands. > > Hmm. This reminds me also that it would be possible with some extra > work to do some command interaction. IMAP supports saving search > results, which can later be accessed with $ parameter. So this could > be made to work: > > doveadm search -s : from foo : fetch text \$ : expunge \$ This is turning into a proper scripting language, so perhaps something like doveadm -e 'search from foo; fetch text $; expunge $' with 'doveadm -F file' to run a script file? Ben From ben at morrow.me.uk Mon Sep 24 20:11:23 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Mon, 24 Sep 2012 18:11:23 +0100 Subject: [Dovecot] Can`t get over 1024 processes on FreeBSD - possible bug? In-Reply-To: <505FFAE7.3070905@um.es> References: <505C3435.30601@blueboard.cz> <505FFAE7.3070905@um.es> Message-ID: <20120924171123.GC63656@anubis.morrow.me.uk> > El 21/09/12 11:32, Tom?? Randa escribi?: > > Hello, > > > > I still cannot get dovecot running with more then 1000 processes, but > > hard limit is 8192 per user in box. I tried everything, including > > modifying startup script of dovecot to set ulimit -u 8192. Could it be > > some dovecot bug or dovecot<>freebsd bug? > > I also tried to set client_limit=2 in imap service to spawn more imap > > clients in one process, but still I am over 1000 processes with kernel > > message: > > > > maxproc limit exceeded by uid 89 You may be running into the kern.maxprocperuid sysctl setting. This is initialized to 9/10ths of kern.maxproc, but can be changed independantly. If you do this you may want to consider setting a default maxproc rlimit in login.conf for the other users on the box. (You may, of course, already have a maxproc limit in login.conf, and that's what's causing the problem, though the default install doesn't include one.) If you have procfs mounted you can check the maxproc rlimit of a running process by looking in /proc/pid/rlimit. In principle it's possible to also get this information with libkvm, but it's not very easy and I don't think any of the standard utilities expose it. Ben From alessio at skye.it Mon Sep 24 20:19:30 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 24 Sep 2012 19:19:30 +0200 Subject: [Dovecot] Dovecot deliver Segmentation fault when arrive the first message In-Reply-To: <50607F04.9040104@skye.it> References: <5059A469.6060604@skye.it> <88D18053-D212-45BF-9E8C-65AA10C7E60F@iki.fi> <5059C2BE.7050006@skye.it> <5059C393.5050209@skye.it> <50607F04.9040104@skye.it> Message-ID: <50609622.7060702@skye.it> Il 24/09/2012 17:40, Alessio Cecchi ha scritto: > Il 19/09/2012 15:07, Alessio Cecchi ha scritto: >> Il 19/09/2012 15:03, Alessio Cecchi ha scritto: >>> Il 19/09/2012 14:48, Timo Sirainen ha scritto: >>>> On 19.9.2012, at 13.54, Alessio Cecchi wrote: >>>> >>>>> LDA is configured and works fine but the problem is when the first >>>>> message arrive "dovecot-lda" return a "Segmentation fault", the >>>>> message is written to the user's Mailbox but the message remains, >>>>> also, in the queue of qmail (deferral: Segmentation_fault/) and at >>>>> the second attempt is delivered fine. >>>> gdb backtrace would be very helpful in figuring out the problem: >>>> http://dovecot.org/bugreport.html >>>> >>> > > Hi Timo, > > had you occasion to see the problem? Can I provide more information? > > Thanks After further testing I found this behavior, a note, I'm using dict quota in mysql. - add a new user - delivery the first email via deliver - "Segmentation fault" - I remove the newly created user - add the same user - delivery the first email via deliver - OK - add a new user - the user connects via pop/imap - delivery the first email via deliver - OK - add a new user - manually create the entry for dict quota <<== - delivery the first email via deliver - OK it seems that if there is user's entry in the dict database the problem does not appear. -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From stsiol at yahoo.co.uk Mon Sep 24 20:42:35 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 18:42:35 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb Message-ID: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Hello all, I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, Xmail and OpenLDAP for a company and serving about 40 acouunts. The machine is wonderful. I am very happy with it. However, I am running out of disk space. It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity has reached 82%.? I am starting of getting nervous. Does anyone know of a painless way to migrate the entire contents directly to another pair of 146Gb SCSI RAID1 disks ? I thought of downtime and using clonezilla, but my last experience with it was questionable. I remember having problems declaring disk re-sizing from the smaller capacity drives to the larger ones. CentOS 5.5 Manual install of : Mysql XMail (pop3/smtp) ASSP (anti spam) Apache / LAMP and last but by no means list : Dovecot Dovecot -n : # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 96 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd Any help would be appreciated or any ideas you might have. Regards, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis? From andrei.michescu at miau.ca Mon Sep 24 21:01:33 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:01:33 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Hello Spyros, As "best practice" you never have the OS and the data/logs/user homes on the same partition or set of disks. If this is the case then your life is pretty easy: -simply create the new set of partitions -mount the new ones in a temporary location -rsync (or copy everything from old partitions) -Stop dovecot / all other daemons that might be using the data -mount the new ones in the place of old ones, mount the old ones in the place of new ones - rsync again (should be quick as not many things changed) - start all your deamons again :P If you do not have separate partitions maybe this is the perfect time to look into that... I would also look into btrfs... might be a good pick for your new partitions. best regards, Andrei > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%.? > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > ? verbose: yes > ? debug: yes > ? debug_passwords: yes > ? passdb: > ? ? driver: passwd-file > ? ? args: /etc/dovecot/passwd > ? passdb: > ? ? driver: pam > ? userdb: > ? ? driver: static > ? ? args: uid=vmail gid=vmail home=/home/vmail/%u > ? userdb: > ? ? driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis? > > !DSPAM:50609d2c301831828332458! > > From robert at schetterer.org Mon Sep 24 21:06:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:06:18 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A11A.9000007@schetterer.org> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > rsync should do the job depending on your whole machine setup it might only be only umount old /home and mount new(bigger) /home after sync ,perhaps with tmp store elsewhere ( for sure you have to have a plan before doing..) but your dovecot is very outdated, i would recommend get up to new hard and software/os install, and then migrate to new machine > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From lists at wildgooses.com Mon Sep 24 21:07:00 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:07:00 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <5060A144.9040607@wildgooses.com> This is one of those questions which is almost too easy if you are familiar with Linux. Trying not to sound like a d*ck, but is it an option to rent someone to help with admin jobs? For example, were it me then I would probably have setup some partitioning scheme with separate partitions for data and operating system? Possibly also using LVM? You have several options, mainly the choice of filesystem will dictate here, but quite possibly you can: 1) Pull the drives one by one and rebuild the raid after each. Keep the old drives since you can technically roll back onto them. Expand the partitions (scary without LVM) and then expand the filesystem on the partitions 2) Boot from a DVD/Flash on your favourite rescue distro (I like sysrecuecd). Create the new raid, copy the old to the new, remove the old drives, reboot from new. Possibly taking the time to repartition and move some data around while you do it (remember to update fstab) Both are fairly simple if you have done it once, but it would be well worth finding someone either local or who will log in via remote control and support you? Final thought: For the size of drives you are looking at, SSD drives are relatively inexpensive and likely comparable with the high end drives you are probably looking to buy? For 40 users I would hazard a guess you likely would be happy with inexpensive low end drives, but certainly a couple of small SSDs will blow away a spinning disk and give you a decent upgrade... Good luck Ed W On 24/09/2012 18:42, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From andrei.michescu at miau.ca Mon Sep 24 21:07:11 2012 From: andrei.michescu at miau.ca (Michescu Andrei) Date: Mon, 24 Sep 2012 14:07:11 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> Message-ID: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Hello Spyros, Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA connector... so the solution below does not really apply to you :( Andrei > Hello Spyros, > > As "best practice" you never have the OS and the data/logs/user homes on > the same partition or set of disks. > > If this is the case then your life is pretty easy: > -simply create the new set of partitions > -mount the new ones in a temporary location > -rsync (or copy everything from old partitions) > -Stop dovecot / all other daemons that might be using the data > -mount the new ones in the place of old ones, mount the old ones in the > place of new ones > - rsync again (should be quick as not many things changed) > - start all your deamons again :P > > If you do not have separate partitions maybe this is the perfect time to > look into that... > > I would also look into btrfs... might be a good pick for your new > partitions. > > best regards, > Andrei > >> Hello all, >> >> I have a DL360 G4 1U server that does a wonderfull job with dovecot >> horde, >> Xmail and OpenLDAP for a company and serving about 40 acouunts. >> >> The machine is wonderful. I am very happy with it. >> However, I am running out of disk space. >> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >> has reached 82%.? >> >> I am starting of getting nervous. >> >> Does anyone know of a painless way to migrate the entire contents >> directly >> to another pair of 146Gb SCSI RAID1 disks ? >> >> I thought of downtime and using clonezilla, but my last experience with >> it >> was questionable. I remember having problems declaring disk re-sizing >> from the smaller capacity drives to the larger ones. >> >> CentOS 5.5 >> Manual install of : >> >> Mysql >> XMail (pop3/smtp) >> ASSP (anti spam) >> Apache / LAMP >> and last but by no means list : Dovecot >> >> Dovecot -n : >> >> # 1.2.16: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >> base_dir: /var/run/dovecot/ >> log_path: /var/log/dovecot/dovecot.log >> info_log_path: /var/log/dovecot/dovecot-info.log >> ssl_parameters_regenerate: 48 >> verbose_ssl: yes >> login_dir: /var/run/dovecot//login >> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >> login_greeting: * Dovecot ready * >> login_max_processes_count: 96 >> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >> mail_plugins: zlib >> auth default: >> ? verbose: yes >> ? debug: yes >> ? debug_passwords: yes >> ? passdb: >> ? ? driver: passwd-file >> ? ? args: /etc/dovecot/passwd >> ? passdb: >> ? ? driver: pam >> ? userdb: >> ? ? driver: static >> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >> ? userdb: >> ? ? driver: passwd >> >> >> Any help would be appreciated or any ideas you might have. >> >> Regards, >> >> spyros >> >> >> >> >> >> >> ---- >> "I merely function as a channel that filters >> music through the chaos of noise" >> - Vangelis? >> >> >> >> > > > > !DSPAM:5060a006309197419291868! > > From robert at schetterer.org Mon Sep 24 21:23:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 24 Sep 2012 20:23:25 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <5060A51D.80404@schetterer.org> Am 24.09.2012 20:07, schrieb Michescu Andrei: > Hello Spyros, > > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( > > Andrei depends how long downtime is acceptable i.e go rsync tmp over usb storage, or simple tmp nfs mounts to other servers are thinkable to minimize downtime should be no big problem its only one server with maildir and less mailboxes and data but thats all much off topic with dovecot -- Best Regards MfG Robert Schetterer From chevalier at dspnet.fr Mon Sep 24 21:41:57 2012 From: chevalier at dspnet.fr (Philippe Chevalier) Date: Mon, 24 Sep 2012 20:41:57 +0200 Subject: [Dovecot] noisy auth-worker messages in logs (dovecot 2.1.8 FreeBSD) In-Reply-To: <20120924150440.GA85969@kyoko.org> References: <20120924134810.GA62723@kyoko.org> <20120924150440.GA85969@kyoko.org> Message-ID: <20120924184157.GA75341@kyoko.org> On Mon, Sep 24, 2012 at 05:04:40PM +0200, Philippe Chevalier wrote: > >I will apply the patch later today and will let you know the result. I applied the patch, and obviously, when getpwnam_r sets the result to NULL and returns EINVAL, dovecot do as if the entry was not found and stays mute. So, thank you, auth is now a lot less noisy. As for the ldap message, it errors if there's no domain in the login. In the doc, it says that %d is empty if there's no domain part. So I guess it's an enhancement request : a configuration option to have it filled out with a default domain if there's no one supplied by the client. Regards, K. -- Kyoko Otonashi's shrine / Le temple de Kyoko Otonashi My tribute to Maison Ikkoku / Mon hommage a Maison Ikkoku Visit http://www.kyoko.org/ From stsiol at yahoo.co.uk Mon Sep 24 21:42:21 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 19:42:21 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> Message-ID: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> ----- Original Message ----- > From: Michescu Andrei > To: Dovecot Mailing List > Cc:? > Sent: Monday, 24 September 2012, 21:07 > Subject: Re: [Dovecot] 76Gb to 146Gb >? > Hello Spyros, >? > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( >? > Andrei >? >> Hello Spyros, >>? >> As "best practice" you never have the OS and the data/logs/user? > homes on >> the same partition or set of disks. >>? >> If this is the case then your life is pretty easy: >> ? -simply create the new set of partitions >> ? -mount the new ones in a temporary location >> ? -rsync (or copy everything from old partitions) >> ? -Stop dovecot / all other daemons that might be using the data >> ? -mount the new ones in the place of old ones, mount the old ones in the >> place of new ones >> ? - rsync again (should be quick as not many things changed) >> ? - start all your deamons again :P >>? >> If you do not have separate partitions maybe this is the perfect time to >> look into that... >>? >> I would also look into btrfs... might be a good pick for your new >> partitions. >>? >> best regards, >> Andrei >>? >>> Hello all, >>>? >>> I have a DL360 G4 1U server that does a wonderfull job with dovecot >>> horde, >>> Xmail and OpenLDAP for a company and serving about 40 acouunts. >>>? >>> The machine is wonderful. I am very happy with it. >>> However, I am running out of disk space. >>> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >>> has reached 82%.? >>>? >>> I am starting of getting nervous. >>>? >>> Does anyone know of a painless way to migrate the entire contents >>> directly >>> to another pair of 146Gb SCSI RAID1 disks ? >>>? >>> I thought of downtime and using clonezilla, but my last experience with >>> it >>> was questionable. I remember having problems declaring disk re-sizing >>> from the smaller capacity drives to the larger ones. >>>? >>> CentOS 5.5 >>> Manual install of : >>>? >>> Mysql >>> XMail (pop3/smtp) >>> ASSP (anti spam) >>> Apache / LAMP >>> and last but by no means list : Dovecot >>>? >>> Dovecot -n : >>>? >>> # 1.2.16: /etc/dovecot/dovecot.conf >>> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >>> base_dir: /var/run/dovecot/ >>> log_path: /var/log/dovecot/dovecot.log >>> info_log_path: /var/log/dovecot/dovecot-info.log >>> ssl_parameters_regenerate: 48 >>> verbose_ssl: yes >>> login_dir: /var/run/dovecot//login >>> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >>> login_greeting: * Dovecot ready * >>> login_max_processes_count: 96 >>> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >>> mail_plugins: zlib >>> auth default: >>> ? verbose: yes >>> ? debug: yes >>> ? debug_passwords: yes >>> ? passdb: >>> ? ? driver: passwd-file >>> ? ? args: /etc/dovecot/passwd >>> ? passdb: >>> ? ? driver: pam >>> ? userdb: >>> ? ? driver: static >>> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >>> ? userdb: >>> ? ? driver: passwd >>>? >>>? >>> Any help would be appreciated or any ideas you might have. >>>? >>> Regards, >>>? >>> spyros >>>? >>>? >>>? >>>? >>>? >>>? >>> ---- >>> "I merely function as a channel that filters >>> music through the chaos of noise" >>> - Vangelis? >>>? >>>? >>>? >>>? >>? >>? >>? >> !DSPAM:5060a006309197419291868! Andrei, Thank you very much for you kind reply and both your messages. Having said that, would it be possible to take away on 72Gb drive (say Drive1 the second drive) and shove in one of the two 146Gb ones ? Shouldn't the array be rebuilt ? Will it use the extra disk space though ? Thanks, spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From lists at wildgooses.com Mon Sep 24 21:55:07 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 24 Sep 2012 19:55:07 +0100 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A144.9040607@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> Message-ID: <5060AC8B.7090703@wildgooses.com> On 24/09/2012 19:07, Ed W wrote: > This is one of those questions which is almost too easy if you are > familiar with Linux. Trying not to sound like a d*ck, but is it an > option to rent someone to help with admin jobs? For example, were it > me then I would probably have setup some partitioning scheme with > separate partitions for data and operating system? Possibly also using > LVM? That came out wrong... What I meant to say was something more like "if you were to employ someone locally they would probably give you a whole bunch of ideas on how you could adjust the setup of the server to be more future proof. It would be worth working with someone just to get that right. For example, here are some ideas that occur to me that you could use ..." Sorry, should re-read my words before hitting send Ed From joe at netmusician.org Mon Sep 24 22:03:46 2012 From: joe at netmusician.org (Joe Auty) Date: Mon, 24 Sep 2012 15:03:46 -0400 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> Message-ID: <5060AE92.5040904@netmusician.org> > > Timo Sirainen > September 24, 2012 10:32 AM > > Well, the good news is that it crashes only after it has already > disconnected the client anyway. But I thought I fixed this bug in > v2.1.10 and I'm not able to reproduce it myself.. Having debugging > information available might show something useful. Try installing > dovecot-dbg package and getting the bt full again? Thanks Timo, I have done so. Here is the results of my debugging info now: gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap-login...done. (no debugging symbols found)...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 hash_table_destroy (_table=0x28) at hash.c:106 106 hash.c: No such file or directory. in hash.c (gdb) bt full #0 hash_table_destroy (_table=0x28) at hash.c:106 table = #1 0x00007ff300721054 in settings_parser_deinit (_ctx=) at settings-parser.c:237 ctx = 0x0 #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=) at master-service-settings-cache.c:86 cache = 0x9f9a60 entry = 0xa016e0 next = 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x00007ff3009a5018 in main_deinit (binary=, argc=2, argv=0x9f8370) at main.c:355 No locals. #4 login_binary_run (binary=, argc=2, argv=0x9f8370) at main.c:407 set_pool = 0x9f8a30 allow_core_dumps = login_socket = c = #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #6 0x0000000000402459 in _start () No symbol table info available. (gdb) > > > Joe Auty > September 23, 2012 7:05 AM >> >> Timo Sirainen >> September 23, 2012 5:58 AM >> >> >> You should have a similar log line about the crash in mail.log (or >> wherever "doveadm log find" says that errors get logged). Find those >> lines, then configure login processes to dump core files. This >> probably should work: >> >> service imap-login { >> executable = imap-login -D >> } >> >> Next time it crashes hopefully you'll have >> /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it >> send it: >> >> gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core >> bt full > > I hope I'm doing this correctly! > > # gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > GNU gdb (GDB) 7.0.1-debian > Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > ... > Reading symbols from /usr/lib/dovecot/imap-login...(no debugging > symbols found)...done. > > warning: Can't read pathname for load map: Input/output error. > Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no > debugging symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 > Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 > Reading symbols from /lib/libc.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.0.9.8 > Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/lib/libcrypto.so.0.9.8 > Reading symbols from /lib/librt.so.1...(no debugging symbols > found)...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/libdl.so.2...(no debugging symbols > found)...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging > symbols found)...done. > Loaded symbols for /lib64/ld-linux-x86-64.so.2 > Reading symbols from /usr/lib/libz.so.1...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libz.so.1 > Reading symbols from /lib/libpthread.so.0...(no debugging symbols > found)...done. > Loaded symbols for /lib/libpthread.so.0 > Core was generated by `dovecot/imap-login ?'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > (gdb) bt full > #0 0x00007f789cd08e14 in hash_table_destroy () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00007f789ccda054 in settings_parser_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #3 0x00007f789cf5e018 in login_binary_run () from > /usr/lib/dovecot/libdovecot-login.so.0 > No symbol table info available. > #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 > No symbol table info available. > #5 0x0000000000402459 in ?? () > No symbol table info available. > #6 0x00007fff8a9c65f8 in ?? () > No symbol table info available. > #7 0x000000000000001c in ?? () > No symbol table info available. > #8 0x0000000000000002 in ?? () > No symbol table info available. > #9 0x00007fff8a9c7e6a in ?? () > No symbol table info available. > #10 0x00007fff8a9c7e7d in ?? () > No symbol table info available. > #11 0x0000000000000000 in ?? () > No symbol table info available. > > >> >> >> Joe Auty >> September 22, 2012 3:57 PM >> Hello, >> >> I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, >> I suspect this might be causing performance issues. Any suggestions >> what I can try to fix this? >> >> >> I'm using the 2.1.10 packages obtained with the following in my >> sources.list: >> >> deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main >> >> >> I need to use Dovecot 2.1.x because I need to support handling >> multiple SSL certs. >> ------------------------------------------------------------------------ > > > > > > Timo Sirainen > September 23, 2012 5:58 AM > > > You should have a similar log line about the crash in mail.log (or > wherever "doveadm log find" says that errors get logged). Find those > lines, then configure login processes to dump core files. This > probably should work: > > service imap-login { > executable = imap-login -D > } > > Next time it crashes hopefully you'll have > /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it: > > gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core > bt full > > > Joe Auty > September 22, 2012 3:57 PM > Hello, > > I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I > suspect this might be causing performance issues. Any suggestions what > I can try to fix this? > > > I'm using the 2.1.10 packages obtained with the following in my > sources.list: > > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main > > > I need to use Dovecot 2.1.x because I need to support handling > multiple SSL certs. > ------------------------------------------------------------------------ From stsiol at yahoo.co.uk Mon Sep 24 22:13:50 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:13:50 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060AC8B.7090703@wildgooses.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A144.9040607@wildgooses.com> <5060AC8B.7090703@wildgooses.com> Message-ID: <1348514030.72104.YahooMailNeo@web132202.mail.ird.yahoo.com> ----- Original Message ----- > From: Ed W > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:55 > Subject: Re: [Dovecot] 76Gb to 146Gb > > On 24/09/2012 19:07, Ed W wrote: >> This is one of those questions which is almost too easy if you are familiar > with Linux.? Trying not to sound like a d*ck, but is it an option to rent > someone to help with admin jobs?? For example, were it me then I would probably > have setup some partitioning scheme with separate partitions for data and > operating system? Possibly also using LVM? > > That came out wrong...? What I meant to say was something more like "if you > were to employ someone locally they would probably give you a whole bunch of > ideas on how you could adjust the setup of the server to be more future proof.? > It would be worth working with someone just to get that right.? For example, > here are some ideas that occur to me that you could use ..." > > Sorry, should re-read my words before hitting send > > Ed > Ed, Don't worry about it. I wasn't offended. I have a lot of experience with linux but not on heavy metal servers. I used to have plenty of experience back in the G2/ G3 era (I was also ACE in the Compaq years) but that was back in the time that Compaq was only supporting Windows OSs and SCO. Also the problem is that I don't have the time to "play" with a spare HP/Compaq server ( I have a couple laying around btw). I'll get round to it at some point. I am just asking you chaps because I am sure people out there had the chance to tinker with newer and better equipment. Thank you for your reply, Best Regards, spyros ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From stsiol at yahoo.co.uk Mon Sep 24 22:24:37 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Mon, 24 Sep 2012 20:24:37 +0100 (BST) Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <5060A11A.9000007@schetterer.org> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> Message-ID: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> ----- Original Message ----- > From: Robert Schetterer > To: dovecot at dovecot.org > Cc: > Sent: Monday, 24 September 2012, 21:06 > Subject: Re: [Dovecot] 76Gb to 146Gb > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >> Hello all, >> > ----%<----%<----%<----%<----%<----%<----%<----%<---- >? > rsync > should do the job > > depending on your whole machine setup it might only be only > umount old /home and mount new(bigger) /home after sync > ,perhaps with tmp store elsewhere > ( for sure you have to have a plan before doing..) > > but your dovecot is very outdated, i would recommend > get up to new hard and software/os install, and then migrate > to new machine > >> > > > -- > Best Regards > MfG Robert Schetterer > On client machines I have thunderbird. What if : 1. I would make sure that thunderbird keeps a local copy of all the message (I think there is a check box somewhere on settings) 2. Make sure all client machines have synced their mailboxes locally on thunderbird. 3. Install a new version of Dovecot/Horde/XMail etc. 4. When the new installation is done, try to sync from the existing clients pc's to dovecot ? Would that work ? It's one scenario I am seriously contemplating. Thank you very much again, s. ? ---- "I merely function as a channel that filters? music through the chaos of noise" - Vangelis From me at junc.org Mon Sep 24 23:59:00 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 22:59:00 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <6ebfff9b6a3d0faeb0a91719e638e232@junc.org> Spyros Tsiolis skrev den 24-09-2012 19:42: > Any help would be appreciated or any ideas you might have. try google "centos cloud server" if you would like to do it local, use all 4 drives with 2 raid1 in the same controller if possible, then use sysrescue cd to tar it all over to the other raid1 while its down no matter how, it will be downtime i am not using centos here so i cant be more specifik http://www.sysresccd.org From me at junc.org Tue Sep 25 00:09:31 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 24 Sep 2012 23:09:31 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <828720e5142c1cb59e96bcd9a15b0efb@junc.org> Spyros Tsiolis skrev den 24-09-2012 20:42: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? this can be done yes, but you will have to do more steps :) first step, remove one drive add the 146 drive wait it for rebuildin when done, remove the last small drive add the last 146 drive wait for it to rebuild now at this stage you have 72g more unused room for new partions make this new partion /home and after its being created, move the user data to it, but this leves 72g system partion with just few gigs needed ?, then i would create the new partion as lwm2, and then possible shrink system, and mount the lwm2 as /home, that way you have more options later if 146 will be to small again warn i have not doing this myself, but if should work in teori atleast From daniel.parthey at informatik.tu-chemnitz.de Tue Sep 25 00:24:48 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Mon, 24 Sep 2012 23:24:48 +0200 Subject: [Dovecot] doveadm with multiple commands In-Reply-To: References: <20120920201741.GA9646@daniel.localdomain> <5DFBB4D5-BA4C-49DE-B8E7-C95982801B9D@iki.fi> Message-ID: <20120924212448.GA9310@daniel.localdomain> Timo Sirainen wrote: > On 21.9.2012, at 8.28, Timo Sirainen wrote: > > >> Timo Sirainen wrote: > >>> doveadm multi [-A | -u wildcards] [ [...]] > >>> > >>> Thoughts? > > > >> As command name I could also think of "doveadm sequence", which > >> implies the commands being executed in serial order. > > > > Hmm. Maybe. > > "sequence" is already commonly used by IMAP protocol and Dovecot code to mean message sequence numbers. I think it would be too confusing to use that word for other things. Ok, so how about "batch"? It reads a series of commands and collects them into one "batch job" which is then carried out. http://en.wikipedia.org/wiki/Batch_(Unix) Regards Daniel -- https://plus.google.com/103021802792276734820 From stan at hardwarefreak.com Tue Sep 25 01:05:35 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 24 Sep 2012 17:05:35 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> Message-ID: <5060D92F.2070106@hardwarefreak.com> On 9/24/2012 1:42 PM, Spyros Tsiolis wrote: > Having said that, would it be possible to take > away on 72Gb drive (say Drive1 the second drive) > and shove in one of the two 146Gb ones ? It's always best to manually take a drive off line before pulling it. > Shouldn't the array be rebuilt ? Depends on how your 6i is configured. Best guess is that it will automatically rebuild the mirror on the new 146GB drive, but... > Will it use the extra disk space though ? It will probably not. You need to read the 6i manual. I sense a hardware upgrade in your near future, either an HP server with 4 bays, or an SFF8088 JBOD chassis and an inexpensive RAID card. You already have the 146GB drives correct? They are HP pluggable drives? Which means they only work in HP gear. If that's the case you need a new server with at least 4 drive bays. You you need to buy an off brand JBOD box and two standard SATA drives. Or maybe your organizations needs more storage on many servers, and it's time to step up to an iSCSI SAN array. -- Stan From michael at orlitzky.com Tue Sep 25 05:59:15 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 22:59:15 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50611E03.3040706@orlitzky.com> On 09/24/2012 01:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. We've done this on the same hardware. You can pick up these servers for cheap; just buy an extra one. Take the new machine, throw two big disks in it, and install Gentoo. Rsync the important stuff. Make sure all of the services are working on the new machine. When you're ready to make the switch, disable external networking on the current live server. Rsync everything again, and then turn the old server off. Add its IP address to the new server. Maybe kick your router's ARP cache to expedite the change. It should only cause a minute or two of downtime. From michael at orlitzky.com Tue Sep 25 06:02:55 2012 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 24 Sep 2012 23:02:55 -0400 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <50611E03.3040706@orlitzky.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <50611E03.3040706@orlitzky.com> Message-ID: <50611EDF.2060602@orlitzky.com> On 09/24/2012 10:59 PM, Michael Orlitzky wrote: > > We've done this on the same hardware. You can pick up these servers for > cheap; just buy an extra one. Take the new machine, throw two big disks > in it, and install Gentoo. I seem to have gone insane, I thought this was on gentoo-user for some reason. Anyway, it's a fine suggestion =) From robert at schetterer.org Tue Sep 25 08:41:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 25 Sep 2012 07:41:26 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <50614406.50400@schetterer.org> Am 24.09.2012 21:24, schrieb Spyros Tsiolis: > > > > > ----- Original Message ----- >> From: Robert Schetterer >> To: dovecot at dovecot.org >> Cc: >> Sent: Monday, 24 September 2012, 21:06 >> Subject: Re: [Dovecot] 76Gb to 146Gb >> >> Am 24.09.2012 19:42, schrieb Spyros Tsiolis: >>> Hello all, >>> >> ----%<----%<----%<----%<----%<----%<----%<----%<---- >> >> rsync >> should do the job >> >> depending on your whole machine setup it might only be only >> umount old /home and mount new(bigger) /home after sync >> ,perhaps with tmp store elsewhere >> ( for sure you have to have a plan before doing..) >> >> but your dovecot is very outdated, i would recommend >> get up to new hard and software/os install, and then migrate >> to new machine >> >>> >> >> >> -- >> Best Regards >> MfG Robert Schetterer >> > > > On client machines I have thunderbird. > > What if : > > 1. I would make sure that thunderbird keeps a local > copy of all the message (I think there is a check box > somewhere on settings) > > 2. Make sure all client machines have synced their > mailboxes locally on thunderbird. > > > 3. Install a new version of Dovecot/Horde/XMail etc. > > 4. When the new installation is done, try to sync > from the existing clients pc's to dovecot ? > > Would that work ? > It's one scenario I am seriously contemplating. > > Thank you very much again, > > s. > in short words ,dont do it like this setup your new server, test it, then do i.e imapsync from old to new switch your ips then ,done only one way which you can go look archives and www for migration tips > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis > -- Best Regards MfG Robert Schetterer From joh.hendriks at gmail.com Tue Sep 25 10:34:02 2012 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Tue, 25 Sep 2012 09:34:02 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <50615E6A.5030109@gmail.com> Spyros Tsiolis schreef: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis A solution you could try is add a new server to the mix. Leave the OS on the DL360 and put the mail data on a second machine. Then use NFS to mount the new data store. If you use a ZFS solution, expanding your data store is a no brainer. 146 GB drives are expensive, for 40 users SATA drives can do the job without any problem. gr Johan Hendriks From voytek at sbt.net.au Tue Sep 25 11:07:23 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Tue, 25 Sep 2012 18:07:23 +1000 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? Message-ID: I have a user who occasionally reads, then deletes emails using his phone, on his and others email accounts, causing emails deleted on mail server, then he says 'oops, can you recover emails I've deleted?' what's the best way to duplicate all emails to duplicate accounts ? backup folders ? so, next time he deletes emails they can be 'recovered' on the server ? I was thinking along the lines of duplicating ALL emails...? any tips and pointers appreciated -- Voytek From Ralf.Hildebrandt at charite.de Tue Sep 25 11:23:53 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 25 Sep 2012 10:23:53 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <20120925082353.GF7796@charite.de> * Voytek Eymont : > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? > > I was thinking along the lines of duplicating ALL emails...? > > any tips and pointers appreciated http://wiki.dovecot.org/Plugins/Lazyexpunge -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From h.reindl at thelounge.net Tue Sep 25 11:24:18 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 25 Sep 2012 10:24:18 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: References: Message-ID: <50616A32.7040808@thelounge.net> Am 25.09.2012 10:07, schrieb Voytek Eymont: > I have a user who occasionally reads, then deletes emails using his phone, > on his and others email accounts, causing emails deleted on mail server, > then he says 'oops, can you recover emails I've deleted?' > > what's the best way to duplicate all emails to duplicate accounts ? backup > folders ? so, next time he deletes emails they can be 'recovered' on the > server ? configure his phone to leave messages on the server if this is the case and he is deleting them manually explain him that it is NOT your problem if he is too stupid to use email at all and sooner or later he will learn it -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From marc at perkel.com Wed Sep 26 00:34:55 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 25 Sep 2012 14:34:55 -0700 Subject: [Dovecot] Dovecot Authentication through different IMAP server? Message-ID: <5062237F.4000000@perkel.com> Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? Here's what I'm thinking. I run a spam filtering service and thinking about a way to give users access to their stored spam on my servers. However real email is forwarded to their server, So I'm wondering if I can configure dovecot somehow to do a back end authentication to their email account to gain access to their spam. Open to suggestions From noel.butler at ausics.net Wed Sep 26 04:23:11 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 26 Sep 2012 11:23:11 +1000 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <5060A11A.9000007@schetterer.org> <1348514677.45861.YahooMailNeo@web132206.mail.ird.yahoo.com> Message-ID: <1348622591.6847.23.camel@tardis> Spyros, Sounds like you have bigger problems, as you appear to have no disaster recovery processes, since your using a standalone server, as recommended to you earlier by Robert, rsync is your friend, and not just now, but daily :) setup a rolling 7 day rsync archive (sata disks are big, cheap and good enough for backups), and dont just rsync the same stuff to the same place every night, thats no good if an error occurs and wipes out your mail, coz rsync will only see it gone, and clean it out of your backup. So long as RedHat have patched dovecot for bugs and still does, 1.2.16 would still be fine, dovecot-1.2.17 is latest 1.2.x series but IIRC no longer supported, but thats fine, there are a lot, and I mean, a LOT of companies and ISP's running that version, moving to 2.x was never going to happen with new releases coming every week or so at one stage, and as 2.1.x doesn't now have nearly as regular updates, I recently myself consider 2.1 finally to be mature enough to put on production servers, which I will likely do around Christmas when its quiet. Lastly, never ever ever rely on your users to have a copy of their mail as a recovery method, it wont work, some will fsck it up, some wont bother, and you will be blamed for not having redundancy inplace. On Mon, 2012-09-24 at 20:24 +0100, Spyros Tsiolis wrote: > > > > ----- Original Message ----- > > From: Robert Schetterer > > To: dovecot at dovecot.org > > Cc: > > Sent: Monday, 24 September 2012, 21:06 > > Subject: Re: [Dovecot] 76Gb to 146Gb > > > > Am 24.09.2012 19:42, schrieb Spyros Tsiolis: > >> Hello all, > >> > > ----%<----%<----%<----%<----%<----%<----%<----%<---- > > > > rsync > > should do the job > > > > depending on your whole machine setup it might only be only > > umount old /home and mount new(bigger) /home after sync > > ,perhaps with tmp store elsewhere > > ( for sure you have to have a plan before doing..) > > > > but your dovecot is very outdated, i would recommend > > get up to new hard and software/os install, and then migrate > > to new machine > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From stan at hardwarefreak.com Wed Sep 26 09:34:55 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 26 Sep 2012 01:34:55 -0500 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> <907ebb208b1c10eaa41446e9daf78231.squirrel@web.miau.ca> <3891667c4e5a0f040c9029cd531e27f9.squirrel@web.miau.ca> <1348512141.2863.YahooMailNeo@web132204.mail.ird.yahoo.com> <5060D92F.2070106@hardwarefreak.com> <1348550991.75699.YahooMailNeo@web132202.mail.ird.yahoo.com> Message-ID: <5062A20F.8060500@hardwarefreak.com> On 9/25/2012 12:29 AM, Spyros Tsiolis wrote: > My clients don't invest in hardware. I live in Greece. Things are really > bad right now. Not that if they were better they (the client) would invest > in their infrastructure. ... > Thank you very much Stan for you kind reply. > > I reply to you in private since I was told to stop the thread > on the dovecot list because it's off-topic. I'm going to bring it back on topic. To recap the issue: 1. 73GB volume, ~82% full. Volume contains OS, spool, and mailboxes 2. Two 146GB drives on hand that are impossible to employ in either a live or offline migration due to hardware limitations--drive slots (0.82*73GB=60GB)-5GB(OS)= ~55GB/40= ~1.375GB/user mailbox (This math assumes the box has already been scoured and things like source files and other temporary stuff has already been removed, along with anything and everything not necessary to the primary functions of this box) 1.375GB/user suggests these users seem to horde old emails, or simply don't keep their mailboxes tidy. Normally I'd guess that a lot of space in a situation like this is taken up by full text search indexes, but you don't appear to have these enabled. I notice in your 'dovecot -n' output that you are already employing zlib, but to what degree is currently unknown to the list. Are you compressing maildirs with a nightly script? If not, and you're compressing on an ad hoc basis, realtime compression during write could save you quite a bit of additional space, extending the operating life of your current drives. Of course a simple request to your users to clean up their mailboxes would surely help as well, if that's a possibility--some users would flog one for making such a suggestion. If you're not fully taking advantage of compression currently, you may want to consider upgrading to Dovecot 2.x which allows transparent compression to dbox and maildir storage. You may want to migrate to mdbox message store format while still using zlib, and enable single instance attachment storage (SIS). As the name suggests, SIS saves an attachment only once, regardless how many user emails in which it exists. See: http://wiki2.dovecot.org/Plugins/Zlib I'd point you to the docs for SIS but I'm unable to find them in the Wiki. Surely someone else can provide the settings and/or update the wiki. -- Stan From robert at schetterer.org Wed Sep 26 09:39:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 08:39:27 +0200 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <5062A31F.9070705@schetterer.org> Am 25.09.2012 23:34, schrieb Marc Perkel: > Just wondering - is there any way for a dovecot server to authenticate > against a different IMAP server? > > Here's what I'm thinking. I run a spam filtering service and thinking > about a way to give users access to their stored spam on my servers. > However real email is forwarded to their server, So I'm wondering if I > can configure dovecot somehow to do a back end authentication to their > email account to gain access to their spam. > > Open to suggestions > i am not sure your idea is the best way to goal your target, running a i.e amavis filter gateway before postfix with some style of quarantaine seems to me a better idea however sasl can do auth against imap i.e ( attention examples may be outdated or different by other distro config style and/or dovecot versions) MECH=?rimap? FLAGS=?-O imap.server.name? http://blog.travisz.com/2010/12/17/debian-mail-server-with-postfix-dovecot-and-saslauthd/ #vim /etc/dovecot/dovecot.conf Look for a line that starts with ?auth default?. Once you fine that line, insert the following: auth default { mechanisms = plain login passdb pam { } userdb passwd { } socket listen { client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix there might better possibilities, wait until more response from the list -- Best Regards MfG Robert Schetterer From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 13:50:44 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 12:50:44 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder Message-ID: <5062DE04.3020601@TU-Cottbus.De> Hi Timo, hi all, we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had configured it in a test environemnt like described in "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" configuration. All actions on first level directories ".INBOX", ."Sent" or ".Archives" are working as expected. If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an error message in dovecot log: Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open expunge mailbox: Character not allowed in mailbox name: '.' What can we do or change to make it work? -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From amateo at um.es Wed Sep 26 13:55:56 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 26 Sep 2012 12:55:56 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server Message-ID: <5062DF3C.3050601@um.es> Hello, I have a infrastructure using director in front of backend servers. So I normally run doveadm commands in director servers, not the backends. The problem I have is when I try to concatenate commands like this: doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w | while read guid uid; do doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} size.physical mailbox-guid $guid uid $uid; done The problem is that although when I run doveadm search command in the backend server I correctly get the list of mails, each line with the mailbox-guid and the uid of the message, when I run the same command in the director server, format of the list is corrupted and there are lines that contains just the mailbox-guid and the next the uid (of the previous) and the mailbox-guid of next, and so on. Like: e62e0d3834ed094e5c7900007efb8a67 66 e62e0d3834ed094e5c7900007efb8a67 71 e62e0d3834ed094e5c7900007efb8a67 74 e62e0d3834ed094e5c7900007efb8a67 75 e62e0d3834ed094e5c7900007efb8a67 77 e62e0d3834ed094e5c7900007efb8a67 78 e62e0d3834ed094e5c7900007efb8a67 so I can't postprocess correctly the output. I think it could be a bug in the communication between the doveadm client and server. PS: As the time of writing this email, I have realized that I could use fetch command directly. But, anyway, I want to report the problem/bug so it can be fixed. From maciej.uhlig at us.edu.pl Wed Sep 26 14:52:18 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 13:52:18 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line Message-ID: <5062EC72.5040505@us.edu.pl> This is a dovecot sieve log line: 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: msgid=: stored mail into mailbox 'INBOX' Would it be possible to have mail size logged here? MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From robert at schetterer.org Wed Sep 26 15:51:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:51:00 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062DE04.3020601@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> Message-ID: <5062FA34.7020502@schetterer.org> Am 26.09.2012 12:50, schrieb Andreas Kossack: > Hi Timo, hi all, > > we want to use the "lazy_expunge" plugin (from dovecot 2.1.10) and had > configured it in a test environemnt like described in > "http://wiki2.dovecot.org/Plugins/Lazyexpunge" (1 namespace) > > There are some subfolders from "INBOX" (like "INBOX/SPAM") in a "maildir++" > configuration. All actions on first level directories ".INBOX", ."Sent" > or ".Archives" are working as expected. > > If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an > error message in dovecot log: > Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open > expunge mailbox: Character not allowed in mailbox name: '.' > > What can we do or change to make it work? > sorry didnt tested lazy_plugin yet try http://wiki2.dovecot.org/Plugins/Listescape Examples Allow '.' characters with Maildir++ layout when virtual hierarchy separator is changed to '/': mail_plugins = $mail_plugins listescape namespace private { separator = / inbox = yes } plugin { # The default escape character is '\', but you can change it. # Note that even here the expansion of % takes place, thus you need to # use "%%" if you want to have the % sign as the escape character. #listescape_char = "\\" } or change your general namespace settings to get it work , might be better choice -- Best Regards MfG Robert Schetterer From robert at schetterer.org Wed Sep 26 15:53:28 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 14:53:28 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAC8.9000108@schetterer.org> Am 26.09.2012 13:52, schrieb Maciej Uhlig: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? > > MU > perhaps try http://wiki2.dovecot.org/Plugins/MailLog plugin { # Events to log. Also available: flag_change save mailbox_create mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Also available: flags vsize from subject mail_log_fields = uid box msgid size } -- Best Regards MfG Robert Schetterer From stephan at rename-it.nl Wed Sep 26 15:53:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 26 Sep 2012 14:53:56 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062EC72.5040505@us.edu.pl> References: <5062EC72.5040505@us.edu.pl> Message-ID: <5062FAE4.1010201@rename-it.nl> Op 9/26/2012 1:52 PM, Maciej Uhlig schreef: > This is a dovecot sieve log line: > > 2012-09-26T13:47:52+02:00 host/192.168.1.1 dovecot: [ID 583609 > mail.info] lmtp(7565, user at domain): vTtJDTrrYlCNHQAAs1WjhA: sieve: > msgid=: stored mail into mailbox 'INBOX' > > Would it be possible to have mail size logged here? You can use the deliver_log_format setting. The default value is: "msgid=%m: %$" I'm not sure where this is documented, so I took a look at the source code to find out which substitutions exist: static struct var_expand_table static_tab[] = { { '$', NULL, NULL }, { 'm', NULL, "msgid" }, { 's', NULL, "subject" }, { 'f', NULL, "from" }, { 'e', NULL, "from_envelope" }, { 'p', NULL, "size" }, { 'w', NULL, "vsize" }, { '\0', NULL, NULL } }; So, this should work for you: deliver_log_format=msgid=%m size=%p: %$ Regards, Stephan From maciej.uhlig at us.edu.pl Wed Sep 26 16:17:36 2012 From: maciej.uhlig at us.edu.pl (Maciej Uhlig) Date: Wed, 26 Sep 2012 15:17:36 +0200 Subject: [Dovecot] mail size needed in dovecot sieve log line In-Reply-To: <5062FAE4.1010201@rename-it.nl> References: <5062EC72.5040505@us.edu.pl> <5062FAE4.1010201@rename-it.nl> Message-ID: <50630070.4040000@us.edu.pl> Stephan Bosch: > > So, this should work for you: > > deliver_log_format=msgid=%m size=%p: %$ > Oh yes it did the trick :-) Thank you and best regards, MU -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3354 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From Andreas.Kossack at TU-Cottbus.De Wed Sep 26 17:15:27 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Wed, 26 Sep 2012 16:15:27 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <5062FA34.7020502@schetterer.org> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> Message-ID: <50630DFF.2050405@TU-Cottbus.De> Hi Robert, >> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >> error message in dovecot log: >> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >> expunge mailbox: Character not allowed in mailbox name: '.' > sorry didnt tested lazy_plugin yet > try > http://wiki2.dovecot.org/Plugins/Listescape It works now with "mail_plugins = $mail_plugins ... lazy_expunge listescape" Thank you for your hint in the right direction. -- Mit freundlichen Gr??en A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ Diese Nachricht wurde aus 100% wiederverwerteten Elektronen erstellt. _______________________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5208 bytes Desc: S/MIME Kryptografische Unterschrift URL: From CMarcus at Media-Brokers.com Wed Sep 26 17:21:07 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 26 Sep 2012 10:21:07 -0400 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50616A32.7040808@thelounge.net> References: <50616A32.7040808@thelounge.net> Message-ID: <50630F53.8060906@Media-Brokers.com> On 2012-09-25 4:24 AM, Reindl Harald wrote: > Am 25.09.2012 10:07, schrieb Voytek Eymont: >> I have a user who occasionally reads, then deletes emails using his phone, >> on his and others email accounts, causing emails deleted on mail server, >> then he says 'oops, can you recover emails I've deleted?' >> >> what's the best way to duplicate all emails to duplicate accounts ? backup >> folders ? so, next time he deletes emails they can be 'recovered' on the >> server ? > configure his phone to leave messages on the server > > if this is the case and he is deleting them manually explain > him that it is NOT your problem if he is too stupid to use > email at all and sooner or later he will learn it Not an option if said user is the boss... -- Best regards, Charles From robert at schetterer.org Wed Sep 26 17:31:41 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 26 Sep 2012 16:31:41 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <506311CD.9060608@schetterer.org> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> Am 25.09.2012 10:07, schrieb Voytek Eymont: >>> I have a user who occasionally reads, then deletes emails using his >>> phone, >>> on his and others email accounts, causing emails deleted on mail server, >>> then he says 'oops, can you recover emails I've deleted?' >>> >>> what's the best way to duplicate all emails to duplicate accounts ? >>> backup >>> folders ? so, next time he deletes emails they can be 'recovered' on the >>> server ? >> configure his phone to leave messages on the server >> >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... > you may do it like this, other soltuions may also thinkable ( depends on your local needs and setup) configure email archive i.e with postfix with i.e always_bcc = address Deliver a copy of all mail to the specified address. In Postfix versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), or pickup(8). sender_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope sender address for an automatic BCC address. This feature is available in Postfix 2.1 and later. recipient_bcc_maps = type:table Search the specified "type:table" lookup table with the envelope recipient address for an automatic BCC address. This feature is available in Postfix 2.1 and later. deliver bc whatever to a second imap account mailbox in a folder set read only via acl i.e with sieve, if you want ,sort by date too etc so if mail was lost ,why ever, the user is able to login into his archive and copy that mail back to his real account -- Best Regards MfG Robert Schetterer From kgc at corp.sonic.net Wed Sep 26 20:34:18 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 10:34:18 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 Message-ID: <50633C9A.5060700@corp.sonic.net> Timo - I upgraded to 2.1.10 on our director servers two nights ago and apart from errors associated with the directors processes restarting everything looked great for ~24 hours until I failed our the real servers last night to update the nfs mount options for the spools. I followed the suggested procedure for each backend server, just run on one of the directors, which seemed to work as expected. doveadm director add x.x.x.x 0 doveadm director flush x.x.x.x The following errors on the directors that started after this went unnoticed until this AM. director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) ... User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. -K From tss at iki.fi Wed Sep 26 20:57:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 20:57:58 +0300 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <50633C9A.5060700@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> Message-ID: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> On 26.9.2012, at 20.34, Kelsey Cummings wrote: > The following errors on the directors that started after this went unnoticed until this AM. > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > ... > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? From kgc at corp.sonic.net Wed Sep 26 21:06:33 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 11:06:33 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> Message-ID: <20120926180633.GE80443@corp.sonic.net> On Wed, Sep 26, 2012 at 08:57:58PM +0300, Timo Sirainen wrote: > On 26.9.2012, at 20.34, Kelsey Cummings wrote: > > > The following errors on the directors that started after this went unnoticed until this AM. > > > > director: User bb host lookup failed: Timeout - queued for 30 secs (Ring synced for 36 secs) > > director: User cc host lookup failed: Timeout - queued for 48 secs (Ring synced for 66 secs, user refreshed 12 secs ago) > > director: User dd host lookup failed: Timeout - queued for 124 secs (Ring synced for 119 secs, weak user, user refreshed 155 secs ago) > > director: User ee host lookup failed: Timeout - queued for 79 secs (Ring synced for 119 secs, weak user, user refreshed 113 secs ago) > > ... > > User ff host lookup failed: Timeout - queued for 30 secs (Ring synced for 7427 secs, weak user, user refreshed 620 secs ago) > > > > This continued, combined with occasional login timeouts (as reported by some internal imap clients.) The login delays/timeouts got bad enough that our load balancers dropped both the servers while I was investigating. They seem to be okay after being restarted. > > After the first few minutes, did all the rest of the error messages contain "weak user" string? Did this happen to a lot of different users (few/some/most)? director_user_expire setting is the default 15 minutes? No, there continued to be a mix of both. The pattern seems to look like this. I'll run some stats later but it looks like a pretty significant number of users where affected. 09:25:21 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5032 secs) 09:25:55 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5066 secs, weak user, user refreshed 64 secs ago) 09:26:28 .. User X host lookup failed: Timeout - queued for 30 secs (Ring synced for 5099 secs, weak user, user refreshed 97 secs ago) -- Kelsey Cummings - kgc at corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407 From tss at iki.fi Wed Sep 26 21:13:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 26 Sep 2012 21:13:26 +0300 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <50630DFF.2050405@TU-Cottbus.De> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> Message-ID: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> On 26.9.2012, at 17.15, Andreas Kossack wrote: >>> If there is an expunge activity on e.g. ".INBOX.SPAM" directory we get an >>> error message in dovecot log: >>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>> expunge mailbox: Character not allowed in mailbox name: '.' > >> sorry didnt tested lazy_plugin yet >> try >> http://wiki2.dovecot.org/Plugins/Listescape > It works now with > "mail_plugins = $mail_plugins ... lazy_expunge listescape" Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd From h.reindl at thelounge.net Wed Sep 26 21:15:24 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 26 Sep 2012 20:15:24 +0200 Subject: [Dovecot] duplicate all emails for 'backup' after user deletes them? In-Reply-To: <50630F53.8060906@Media-Brokers.com> References: <50616A32.7040808@thelounge.net> <50630F53.8060906@Media-Brokers.com> Message-ID: <5063463C.9090002@thelounge.net> Am 26.09.2012 16:21, schrieb Charles Marcus: > On 2012-09-25 4:24 AM, Reindl Harald wrote: >> if this is the case and he is deleting them manually explain >> him that it is NOT your problem if he is too stupid to use >> email at all and sooner or later he will learn it > > Not an option if said user is the boss... surely even the boss has to understand that there is no technical solution for a social problem - if not: wrong company -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 259 bytes Desc: OpenPGP digital signature URL: From stsiol at yahoo.co.uk Thu Sep 27 00:19:06 2012 From: stsiol at yahoo.co.uk (Spyros Tsiolis) Date: Wed, 26 Sep 2012 22:19:06 +0100 (BST) Subject: [Dovecot] second dovecot imap server Message-ID: <1348694346.16608.YahooMailNeo@web132206.mail.ird.yahoo.com> Hello again all, I would like to thank everyone for rushing to help me with the dovecot migration issue. I am , currently, reading and studying what has been said by all the good and kind people who responded to? this list. OK, on the same issue,I thought of adding a totally new server with dovecot acting as a secondary/backup server for the mail servicing. Would that be possible ?? What if I manage to make it work; Can I retire the old(er) server ? I would think that this is more on-topic than my last post. If I am out of line again, please respond in private. Thank you all! spyros ? ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis From kgc at corp.sonic.net Thu Sep 27 00:38:37 2012 From: kgc at corp.sonic.net (Kelsey Cummings) Date: Wed, 26 Sep 2012 14:38:37 -0700 Subject: [Dovecot] (new) director issues in 2.1.10 In-Reply-To: <20120926180633.GE80443@corp.sonic.net> References: <50633C9A.5060700@corp.sonic.net> <94338BE3-A529-4A38-92F0-0F6CA9A14547@iki.fi> <20120926180633.GE80443@corp.sonic.net> Message-ID: <506375DD.3080804@corp.sonic.net> On 09/26/12 11:06, Kelsey Cummings wrote: > No, there continued to be a mix of both. The pattern seems to look like > this. I'll run some stats later but it looks like a pretty significant > number of users where affected. Timo, it looks like the total number of affected users was only about 250 and that most of their erred connections were surrounded by successful sessions. -K From eliezer at ngtech.co.il Thu Sep 27 01:25:07 2012 From: eliezer at ngtech.co.il (Eliezer Croitoru) Date: Thu, 27 Sep 2012 00:25:07 +0200 Subject: [Dovecot] 76Gb to 146Gb In-Reply-To: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> References: <1348508555.86315.YahooMailNeo@web132203.mail.ird.yahoo.com> Message-ID: <506380C3.8010903@ngtech.co.il> On 9/24/2012 7:42 PM, Spyros Tsiolis wrote: > Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot It really depends on the raid you have. is it software or hardware raid? if it's software raid it will be more then simple to do it while taking one drive out put a new one in and use let say FINNIX linux from cd\dvd\usb and manage the whole partitioning copying etc from another OS while not harming anything in the old HDDs. you will might need to setup a new partitions manually on the new drive but just make a plan try it on some small VM to make sure the steps you are doing are fine for centos 5.5 and go for it. rsync is your friend!! in this case. someone mentioned here Gentoo which one installation of the system can give you very big backgorund on manual partitioning chrooting and other basic stuff that can help you in the process. Regards, Eliezer > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis From amateo at um.es Thu Sep 27 08:52:59 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 27 Sep 2012 07:52:59 +0200 Subject: [Dovecot] bug formatting results when using doveadm-server In-Reply-To: <5062DF3C.3050601@um.es> References: <5062DF3C.3050601@um.es> Message-ID: <5063E9BB.8010104@um.es> I forgot to mention that I'm running dovecot 2.1.9 in my servers. El 26/09/12 12:55, Angel L. Mateo escribi?: > Hello, > > I have a infrastructure using director in front of backend servers. > So I normally run doveadm commands in director servers, not the > backends. The problem I have is when I try to concatenate commands like > this: > > doveadm search -S /var/run/dovecot/auth-userdb -u ${user} SAVEDSINCE 5w > | while read guid uid; do > doveadm fetch -S /var/run/dovecot/auth-userdb -u ${user} > size.physical mailbox-guid $guid uid $uid; > done > > The problem is that although when I run doveadm search command in > the backend server I correctly get the list of mails, each line with the > mailbox-guid and the uid of the message, when I run the same command in > the director server, format of the list is corrupted and there are lines > that contains just the mailbox-guid and the next the uid (of the > previous) and the mailbox-guid of next, and so on. Like: > > e62e0d3834ed094e5c7900007efb8a67 66 > e62e0d3834ed094e5c7900007efb8a67 71 > e62e0d3834ed094e5c7900007efb8a67 74 > e62e0d3834ed094e5c7900007efb8a67 > 75 e62e0d3834ed094e5c7900007efb8a67 > 77 e62e0d3834ed094e5c7900007efb8a67 > 78 e62e0d3834ed094e5c7900007efb8a67 > > so I can't postprocess correctly the output. I think it could be a > bug in the communication between the doveadm client and server. > > PS: As the time of writing this email, I have realized that I could use > fetch command directly. But, anyway, I want to report the problem/bug so > it can be fixed. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From Jost.Krieger+dovecot at rub.de Thu Sep 27 12:08:43 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 11:08:43 +0200 Subject: [Dovecot] Changing Quota downward for special Mailboxes? (2.1.6) In-Reply-To: <20120919125241.GF2010@ruhr-uni-bochum.de> References: <20120918114011.GH8536@ruhr-uni-bochum.de> <20120919125241.GF2010@ruhr-uni-bochum.de> Message-ID: <20120927090843.GW12477@ruhr-uni-bochum.de> On Wed 19 Sep 2012 02:52:41 PM GMT, Jost Krieger wrote: > Oops, that's not what I want! > > How about another command line parameter for lda to suppress the > emergency save? We know exactly when we are trying to save probable > spam. I've tried my luck. Would the attached patch be sensible and welcome? Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | -------------- next part -------------- diff -r 88a05f387743 doc/man/dovecot-lda.1.in --- a/doc/man/dovecot-lda.1.in Thu Sep 27 02:55:14 2012 +0300 +++ b/doc/man/dovecot-lda.1.in Thu Sep 27 11:04:56 2012 +0200 @@ -5,7 +5,7 @@ .\"------------------------------------------------------------------------ .SH SYNOPSIS .B dovecot\-lda -.RB [ \-ek ] +.RB [ \-eEk ] [\fB\-a\fP \fIaddress\fP] [\fB\-c\fP \fIconfig_file\fP] [\fB\-d\fP \fIusername\fP] @@ -61,6 +61,10 @@ The default is to send a rejection mail ourself. .\"------------------------------------- .TP +.B \-E +If mail delivery fails, don't try to do an emergency delivery attempt to INBOX. +.\"------------------------------------- +.TP .BI \-f\ envelope_sender Envelope sender address. .\"------------------------------------- diff -r 88a05f387743 src/lda/main.c --- a/src/lda/main.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lda/main.c Thu Sep 27 11:04:56 2012 +0200 @@ -310,7 +310,7 @@ master_service = master_service_init("lda", MASTER_SERVICE_FLAG_STANDALONE | MASTER_SERVICE_FLAG_DONT_LOG_TO_STDERR, - &argc, &argv, "a:d:ef:km:p:r:"); + &argc, &argv, "a:d:eEf:km:p:r:"); memset(&ctx, 0, sizeof(ctx)); ctx.session = mail_deliver_session_init(); @@ -334,6 +334,9 @@ case 'e': stderr_rejection = TRUE; break; + case 'E': + ctx.suppress_emergency_delivery = TRUE; + break; case 'f': /* envelope sender address */ ctx.src_envelope_sender = diff -r 88a05f387743 src/lib-lda/mail-deliver.c --- a/src/lib-lda/mail-deliver.c Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.c Thu Sep 27 11:04:56 2012 +0200 @@ -402,7 +402,7 @@ ret = mail_deliver_save(ctx, ctx->dest_mailbox_name, 0, NULL, storage_r); } - if (ret < 0 && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { + if (ret < 0 && !ctx->suppress_emergency_delivery && strcasecmp(ctx->dest_mailbox_name, "INBOX") != 0) { /* still didn't work. try once more to save it to INBOX. */ ret = mail_deliver_save(ctx, "INBOX", 0, NULL, storage_r); diff -r 88a05f387743 src/lib-lda/mail-deliver.h --- a/src/lib-lda/mail-deliver.h Thu Sep 27 02:55:14 2012 +0300 +++ b/src/lib-lda/mail-deliver.h Thu Sep 27 11:04:56 2012 +0200 @@ -49,6 +49,7 @@ struct var_expand_table *var_expand_table; bool tried_default_save; + bool suppress_emergency_delivery; bool saved_mail; bool save_dest_mail; /* Delivery failed because user is out of quota / disk space */ From davide.marchi at mail.cgilfe.it Thu Sep 27 12:22:47 2012 From: davide.marchi at mail.cgilfe.it (Davide) Date: Thu, 27 Sep 2012 11:22:47 +0200 Subject: [Dovecot] Antispam plugin problem (CRM114) Message-ID: <50641AE7.6040201@mail.cgilfe.it> Hi to all, sorry in advance for my poor english, this is the first time that i wrote to a list if i make mistake .... excuseme. My problem is this: i have dovecot 2.1.8 installed and functioning from 2 years one week ago i have installed crm114 for my last spam detection filter "version 20100106-BlameMichelson (TRE 0.8.0 (BSD))" My mail system is qmail that through .qmail default correctly tag with X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-27CA1CFB X-CRM114-CacheID: sfid-20120927_105129_798028_B0035817 X-CRM114-Status: GOOD ( 28.64 ) headers dovecot.conf is as follow: plugin { . . . antispam_signature = X-CRM114-CacheID antispam_verbose_debug = 1 antispam_debug_target = stderr antispam_backend = crm114 antispam_signature_missing = error antispam_trash = Trash antispam_spam = SPAM antispam_unsure = UNSURE antispam_crm_binary = /opt/crm114/mailreaver.crm antispam_crm_args = --fileprefix=/opt/crm114/ . . . } protocol imap { . . . mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = $mail_plugins quota imap_quota mail_log antispam . . . } I have a sieve default compiled script that automatically put mails in SPAM folder or UNSURE folder if respectively in message hedere there is the evaluation of crm114 (UNSURE or SPAM). When i move mails incorrectly tagged by crm (for example from SPAM to my inbox or from UNSURE to SPAM) thunderbird (company official mail client ver 15.01) go in error with the following message: "Operation failed over folder 'UNSURE'. Server for account davide.marchi at mail.cgilfe.it said: [CANNOT] Failed to call crm114 binary.." I dont'know where is my mistake if someone can help me is much appreciated Thank you -- */Davide Marchi /Teorema Ferrara Srl /(Tel: /**/+39 0532 783161)/**/ (Fax: +/**/39 0532 783368/**/)/**//**/ /**//**/Davide.Marchi at mail.cgilfe.it /CONFIDENZIALITA? *** Questo messaggio, inclusi gli eventuali allegati, ? indirizzato solo ai destinatari e pu? contenere informazioni riservate e confidenziali. Se avete ricevuto il messaggio senza esserne un destinatario, siete pregati di non utilizzare, copiare o distribuire questo messaggio o i suoi allegati. Se avete ricevuto il messaggio per errore, siete pregati di cancellarlo dal vostro sistema, assieme a tutti gli allegati, e di informare immediatamente il mittente,sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. *CONFIDENTIALITY*** This message and any attachments are intended only for the use of the addressee(s) and may contain information that is privileged and/or confidential. If the reader of the message is not the intended recipient(s) or an authorized representative of the intended recipient(s), please do not use, copy, distribute this email or its attachments or take action based on them. If you have received this communication in error, please notify us immediately by email and delete the message and any attachments from your system. *Teorema FerraraSrl - Via Spronello, 7 - 44121 Ferrara Italia - P. IVA 01244490387** P**S**tampa questo messaggio solo se veramente necessario, pensa alla natura! - Please consider whether it is necessary to print this e-mail* From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 14:41:15 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 13:41:15 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox Message-ID: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Hello. After we switched the format from mbox to mdbox on our dovecot (version 2.1.9) servers last week, some of our users were missing the subfolders of their inboxes. So far we thought that with the mbox format it is not possible to have mail folders containing both messages and subfolders. Now we know that there is one exception from this rule: the inbox. With mbox format dovecot supports subfolders of the inbox, and a few of our users actually detected and used this feature! Internally the messages are stored in /var/mail/user or in a file named 'inbox' and the subfolders are in the directory 'INBOX'. (The OS is case sensitiv.) The problem was that doveadm and dsync do not support subfolders of the inbox and therefore these folders did not get synchronized from our old mbox server to the new mdbox dovecot server. The command doveadm -o mail_location=mbox:~user123/Mail mailbox list -u user123 '*' did not show any subfolder of INBOX, whereas the the following with a wrong mail_location showed that there are subfolders of INBOX doveadm -o mail_location=mbox:~user123 mailbox list -u user123 '*' Finally we could import the missing subfolders one after one in the mdbox server with commands like this: doveadm import -u user123 mbox:~user123/Mail "" mailbox INBOX/subfolder all It would be nice if doveadm could be changed to not overlook subfolders of the inbox with mbox format. Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From Jost.Krieger+dovecot at rub.de Thu Sep 27 15:13:13 2012 From: Jost.Krieger+dovecot at rub.de (Jost Krieger) Date: 27 Sep 2012 14:13:13 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> Message-ID: <20120927121131.GC12477@ruhr-uni-bochum.de> On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > After we switched the format from mbox to mdbox on our dovecot > (version 2.1.9) servers last week, some of our users were missing > the subfolders of their inboxes. So far we thought that with the > mbox format it is not possible to have mail folders containing both > messages and subfolders. Now we know that there is one exception > from this rule: the inbox. With mbox format dovecot supports > subfolders of the inbox, and a few of our users actually detected > and used this feature! Please see the thread "No status for INBOX/* in mbox NS (2.1.6)" there's a patch from Timo. Yours Jost Krieger -- | Jost.Krieger+sig at ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 | From Juergen.Obermann at hrz.uni-giessen.de Thu Sep 27 15:56:07 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Thu, 27 Sep 2012 14:56:07 +0200 Subject: [Dovecot] with mbox format doveadm ignores subfolders of inbox In-Reply-To: <20120927121131.GC12477@ruhr-uni-bochum.de> References: <20120927134115.19081j2n3levbypw@webmail.hrz.uni-giessen.de> <20120927121131.GC12477@ruhr-uni-bochum.de> Message-ID: <20120927145607.89971mht64oiixnk@webmail.hrz.uni-giessen.de> Am Do, 27 Sep 2012 schrieb Jost Krieger: > On Thu 27 Sep 2012 01:41:15 PM GMT, J?rgen Obermann wrote: > >> After we switched the format from mbox to mdbox on our dovecot >> (version 2.1.9) servers last week, some of our users were missing >> the subfolders of their inboxes. So far we thought that with the >> mbox format it is not possible to have mail folders containing both >> messages and subfolders. Now we know that there is one exception >> from this rule: the inbox. With mbox format dovecot supports >> subfolders of the inbox, and a few of our users actually detected >> and used this feature! > > Please see the thread > "No status for INBOX/* in mbox NS (2.1.6)" > there's a patch from Timo. Thanks for the info. This patch seems to be in dovecot 2.1.10, we made our format switch from mbox to mdbox with version 2.1.9. Obviously one week too early ;-). Greetings, J?rgen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From blevi.linux at gmail.com Thu Sep 27 16:25:34 2012 From: blevi.linux at gmail.com (Birta Levente) Date: Thu, 27 Sep 2012 16:25:34 +0300 Subject: [Dovecot] segfault in Debian Squeeze + Dovecot 2.1.10 In-Reply-To: <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> References: <505E180F.5060407@netmusician.org> <505EED00.6090109@netmusician.org> <50607456.1040709@gmail.com> <7362A21F-48A4-4D6C-A351-F97B42874695@iki.fi> Message-ID: <506453CE.7000608@gmail.com> On 24/09/2012 17:58, Timo Sirainen wrote: > On 24.9.2012, at 17.55, Birta Levente wrote: > >> On 24/09/2012 17:32, Timo Sirainen wrote: >>> On 23.9.2012, at 14.05, Joe Auty wrote: >>> >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> (gdb) bt full >>>> #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>>> No symbol table info available. >>>> #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 >>> >>> Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again? >>> >> >> I have the same problem, but on centos 6.3 64bit. How can I give you the debug information? > > Show your doveconf -n output at least. As for debugging information, that would depend on how you installed Dovecot? From some RPM or sources? > Hi Timo I just want to ask you: this issue is still in your task list? If you need more debug information please tell me how can I give you. Thanks, Levi From humberto.pedroso at gmail.com Thu Sep 27 17:36:09 2012 From: humberto.pedroso at gmail.com (Humberto A. Pedroso) Date: Thu, 27 Sep 2012 11:36:09 -0300 Subject: [Dovecot] Deleted to trash plugin Message-ID: Hello, Does anybody are using deleted_to_trash plugin with Dovecot 2.1.9? I'm trying to compile and I'm with some difficulties. I tested with dovecot 2.0 and it works very well. -- Humberto From bob at computerisms.ca Thu Sep 27 19:37:20 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 09:37:20 -0700 Subject: [Dovecot] sieve vacation Message-ID: <1348763840.533.187.camel@worklian> Hello, I set up pigeonhole on a server and am using roundcube's sieverules to create a set of rules. When using the vacation feature, I can see in the logs that dovecot is sending the vacation response, but the recipient never receives it. What method does dovecot use to send the response? Does it use the sendmail binary, does it use the mail command, does it connect to some smtp server somewhere, does it use qmail-inject if it finds it? I find no evidence it is using any of these methods on my server, but I am not sure if one of these methods is failing or if there is some place else I should be looking for the missing vacation response... Thank you for any suggestions... -- Computerisms Bob Miller 867-334-7117 / 867 633 3760 http://computerisms.ca From robert at schetterer.org Thu Sep 27 19:52:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 18:52:29 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348763840.533.187.camel@worklian> References: <1348763840.533.187.camel@worklian> Message-ID: <5064844D.2000502@schetterer.org> Am 27.09.2012 18:37, schrieb Bob Miller: > the logs that dovecot is sending the vacation response, but the > recipient never receives it. show the logs study http://wiki2.dovecot.org/LDA Common configuration The settings are listed in the example conf.d/15-lda.conf file. The important settings are: postmaster_address is used as the From: header address in bounce mails hostname is used in generated Message-IDs and in Reporting-UA: header in bounce mails sendmail_path is used to send mails. Note that the default is /usr/sbin/sendmail, which doesn't necessarily work the same as /usr/lib/sendmail. Alternatively you can use submission_host to send mails via the specified SMTP server. auth_socket_path specifies the UNIX socket to auth-userdb where LDA can lookup userdb information when -d parameter is used. See below how to configure Dovecot to configure the socket. -- Best Regards MfG Robert Schetterer From bob at computerisms.ca Thu Sep 27 20:12:51 2012 From: bob at computerisms.ca (Bob Miller) Date: Thu, 27 Sep 2012 10:12:51 -0700 Subject: [Dovecot] sieve vacation In-Reply-To: <5064844D.2000502@schetterer.org> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> Message-ID: <1348765971.533.189.camel@worklian> Robert, > The settings are listed in the example conf.d/15-lda.conf file. The > important settings are: > > sendmail_path is used to send mails. Note that the default is > /usr/sbin/sendmail, which doesn't necessarily work the same as > /usr/lib/sendmail. > > Alternatively you can use submission_host to send mails via the > specified SMTP server. This is the missing piece of information I wasn't finding. Thank you very much... From robert at schetterer.org Thu Sep 27 21:11:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 27 Sep 2012 20:11:00 +0200 Subject: [Dovecot] sieve vacation In-Reply-To: <1348765971.533.189.camel@worklian> References: <1348763840.533.187.camel@worklian> <5064844D.2000502@schetterer.org> <1348765971.533.189.camel@worklian> Message-ID: <506496B4.4000508@schetterer.org> Am 27.09.2012 19:12, schrieb Bob Miller: > Robert, > > >> The settings are listed in the example conf.d/15-lda.conf file. The >> important settings are: >> >> sendmail_path is used to send mails. Note that the default is >> /usr/sbin/sendmail, which doesn't necessarily work the same as >> /usr/lib/sendmail. >> >> Alternatively you can use submission_host to send mails via the >> specified SMTP server. > > This is the missing piece of information I wasn't finding. Thank you > very much... > youre welcome Bob *g -- Best Regards MfG Robert Schetterer From jbates at brightok.net Thu Sep 27 23:30:18 2012 From: jbates at brightok.net (Jack Bates) Date: Thu, 27 Sep 2012 15:30:18 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused Message-ID: <5064B75A.7060307@brightok.net> Version: 2.1.9 Connecting directly to the proxy server on the LMTP port, issue lhlo, mail from, rcpt to, data. After the . closing of data, it just hangs there. The logs on the proxy server show: Sep 27 19:55:12 proxy1 dovecot: lmtp(9398): Connect from 69.8.2.71 Sep 27 19:55:24 proxy1 dovecot: lmtp(9398): Error: lmtp client: connect(lmtp, 7025) failed: Connection refused root 9398 1102 0 19:53 ? 00:00:00 dovecot/lmtp The proxy server is hard coded (not director in this case). I purposefully shut it down to see how the system would handle failures. Unfortunately, it appears that the process just hangs there. For fun, I tried a separate protocol lmtp option in the config to try proxy_timeout. It made no difference. Missing some error handling? protocol lmtp { passdb { driver = static args = proxy=y host=%s nopassword=y proxy_timeout=120 } } Jack From Andreas.Kossack at TU-Cottbus.De Fri Sep 28 10:25:55 2012 From: Andreas.Kossack at TU-Cottbus.De (Andreas Kossack) Date: Fri, 28 Sep 2012 09:25:55 +0200 Subject: [Dovecot] Plugin "lazy_expunge" and subfolder In-Reply-To: <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> References: <5062DE04.3020601@TU-Cottbus.De> <5062FA34.7020502@schetterer.org> <50630DFF.2050405@TU-Cottbus.De> <37085055-290C-414F-B40E-E8932B3A0E81@iki.fi> Message-ID: <50655103.5090205@TU-Cottbus.De> Hi Timo, hi all, >>>> Sep 26 12:27:55 imap(testuser): Error: lazy_expunge: Couldn't open >>>> expunge mailbox: Character not allowed in mailbox name: '.' >> >>> try >>> http://wiki2.dovecot.org/Plugins/Listescape >> It works now with >> "mail_plugins = $mail_plugins ... lazy_expunge listescape" > > Ugh, kludgy. Here's a proper fix: http://hg.dovecot.org/dovecot-2.1/rev/4e431b202cfd Now it works lika a charm without "listescape" "mail_plugins = $mail_plugins ... lazy_expunge" Thank you Timo... -- Best regards A. Kossack _______________________________________________________________________________ Andreas Kossack Andreas.Kossack at TU-Cottbus.De BTU Cottbus, Brandenburg, Germany, Walther-Pauer-Str.2, 03046 Cottbus http://wwwca.telesec.de/cgi-bin/caservice/Common/InstallRoot/DT-Root-CA-2.der _______________________________________________________________________________ From jbates at brightok.net Fri Sep 28 19:37:50 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 11:37:50 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5064B75A.7060307@brightok.net> References: <5064B75A.7060307@brightok.net> Message-ID: <5065D25E.1030507@brightok.net> On 9/27/2012 3:30 PM, Jack Bates wrote: > Version: 2.1.9 > > Connecting directly to the proxy server on the LMTP port, issue lhlo, > mail from, rcpt to, data. After the . closing of data, it just hangs > there. The logs on the proxy server show: > Looking at dovecot's proxy code. Calls to connect_connect_ip do not return an error when the server isn't there. This may be expected, but it is what determines failure/OK in the LMTP code. The log entry is performed by lmtp_client_wait_connect callback. In the standard client-common-auth.c code, the error returned for a failure is 1) syslog error is reported by proxy_wait_connect() callback 2) error to client is returned by an error in proxy_input() In the lmtp code, we issue a "250 2.1.5 OK". Right after, the lmtp_client_wait_connect() callback is called and issues the connection refused error. However, we've already approved the rcpt. I am not sure which method we really want in the lmtp proxy. Should it be failing at the initial rcpt command or after the data command. Currently, though I haven't checked yet, we don't appear to be detecting the input failure and handling the data command correctly since we've already allowed the rcpt to proceed without a backend proxy available. Jack From jbates at brightok.net Fri Sep 28 20:44:21 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 12:44:21 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065D25E.1030507@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> Message-ID: <5065E1F5.4010506@brightok.net> On 9/28/2012 11:37 AM, Jack Bates wrote: > On 9/27/2012 3:30 PM, Jack Bates wrote: >> Version: 2.1.9 >> >> Connecting directly to the proxy server on the LMTP port, issue lhlo, >> mail from, rcpt to, data. After the . closing of data, it just hangs >> there. The logs on the proxy server show: > I am not sure which method we really want in the lmtp proxy. Should it > be failing at the initial rcpt command or after the data command. > Currently, though I haven't checked yet, we don't appear to be > detecting the input failure and handling the data command correctly > since we've already allowed the rcpt to proceed without a backend > proxy available. Further testing shows that the connection fails are normally handled after the DATA block. It works perfectly if you only fail one recipient. If all recipients are failed, the code locks up instead of telling the client that they are all failed. Now to find out where between lmtp_client_fail() and some other part of the code, we aren't treating it right. :( Jack From tss at iki.fi Fri Sep 28 21:59:52 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 21:59:52 +0300 Subject: [Dovecot] Hanging IMAP sessions on Mac OS X with dovecot 2.1.10 - worked fine with 2.0.15 In-Reply-To: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> References: <73664788-ACE6-4175-AB26-8F2877391870@kiez.net> Message-ID: <43585454-3155-4B89-B5FD-2D2ACA80E562@iki.fi> On 22.9.2012, at 19.11, Clemens Schrimpe wrote: > Well, that led my to believe, that there was something wrong with the -so called- "mach bootstrap context". I usually start dovecot from with a (home-brewn) startup-script, which invokes it (practically) like so: > > sudo /usr/libexec/StartupItemContext dovecot > > (again: all this was working fine under 2.0.15) > > Now with 2.1.10, when I manually invoke dovecot with just > > sudo dovecot > > Everything appears to work fine - at least the sessions don't get stuck any more. But as soon as I logout (with dovecot still running in the background) it loses it's "mach bootstrap context" and finds itself unable to perform even the simplest tasks, like mapping a username to a uid, etc. pp. -- so starting it without the "/usr/libexec/StartupItemContext" in the background is out of the question. I don't really know about mach contexts or how they're supposd to work.. > What has changed with regards to "processual context" between 2.0.15 and 2.1.10 when the "imap" process is spawned/exec'd? Any environmental cleanups, closing of unknown fds, deletion/modification of environment variables, process-group-handlers, etc.? Not much I think. I just looked at the diff between 2.0 and 2.1 and don't really see anything I could blame. If you have time you could try bisecting with mercurial (basically try different versions from hg) to isolate the change that broke it. I don't really have time to debug this.. From tss at iki.fi Fri Sep 28 22:01:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:01:13 +0300 Subject: [Dovecot] Dovecot Clustering with dsync over ssh In-Reply-To: References: Message-ID: On 23.9.2012, at 10.56, weber at zackbummfertig.de wrote: > i only found an old post of tiemo where he announced that he will do an wiki article > how to setup dovecot cluster with dsync over ssh. > anyone knows if this artcile exists and where? > > i would like to setup a dovecot cluster with 2 machines. There are some mailing list posts about how to do it. Anyway I don't advertise it much yet because v2.1's dsync isn't really quite good enough to do it well. v2.2 dsync should work well. From tss at iki.fi Fri Sep 28 22:02:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 28 Sep 2012 22:02:42 +0300 Subject: [Dovecot] Dovecot Authentication through different IMAP server? In-Reply-To: <5062237F.4000000@perkel.com> References: <5062237F.4000000@perkel.com> Message-ID: <9952E585-05B8-4418-A3BD-0ED0F0A84165@iki.fi> On 26.9.2012, at 0.34, Marc Perkel wrote: > Just wondering - is there any way for a dovecot server to authenticate against a different IMAP server? v2.1 has passdb imap From jbates at brightok.net Fri Sep 28 23:12:44 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:12:44 -0500 Subject: [Dovecot] LTMP Proxy fails when backend server connection refused In-Reply-To: <5065E1F5.4010506@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> Message-ID: <506604BC.6050503@brightok.net> On 9/28/2012 12:44 PM, Jack Bates wrote: > On 9/28/2012 11:37 AM, Jack Bates wrote: >> On 9/27/2012 3:30 PM, Jack Bates wrote: >>> Version: 2.1.9 >>> >>> Connecting directly to the proxy server on the LMTP port, issue >>> lhlo, mail from, rcpt to, data. After the . closing of data, it just >>> hangs there. The logs on the proxy server show: >> > Further testing shows that the connection fails are normally handled > after the DATA block. It works perfectly if you only fail one > recipient. If all recipients are failed, the code locks up instead of > telling the client that they are all failed. > > Now to find out where between lmtp_client_fail() and some other part > of the code, we aren't treating it right. :( It appears that the callbacks for the proxy connections are handled in the ioloop stuff. If you have at least one valid proxy session open, then proxy_try_finish will get called and all the replies are sent. The problem appears to be if all backend proxy sessions are bad (ie single recipient, proxy server down), there is nothing for ioloop to callback to. proxy_try_finish will never get called, and we end up locking up at epoll_wait as there are no events for us to process. Code needs to be written to handle the special case of us not having any proxy callbacks as they are all bad. Jack From jbates at brightok.net Fri Sep 28 23:29:11 2012 From: jbates at brightok.net (Jack Bates) Date: Fri, 28 Sep 2012 15:29:11 -0500 Subject: [Dovecot] LTMP Proxy failure fix/hack In-Reply-To: <506604BC.6050503@brightok.net> References: <5064B75A.7060307@brightok.net> <5065D25E.1030507@brightok.net> <5065E1F5.4010506@brightok.net> <506604BC.6050503@brightok.net> Message-ID: <50660897.6040008@brightok.net> On 9/28/2012 3:12 PM, Jack Bates wrote: > > Code needs to be written to handle the special case of us not having > any proxy callbacks as they are all bad. > Timo, please check and approve. This was diff'd on 2.1.10 on my test server (2.1.9 and 2.1.10 at least had this callback issue). *** lmtp-proxy.c-orig 2012-09-28 20:17:36.138916678 +0000 --- lmtp-proxy.c 2012-09-28 20:18:12.241940780 +0000 *************** *** 300,303 **** --- 300,304 ---- lmtp_client_send(conn->client, conn->data_input); lmtp_client_send_more(conn->client); } + lmtp_proxy_try_finish(proxy); } ie, call lmtp_proxy_try_finish once. If all is bad, this will wrap us up. If we have valid proxies, it'll probably not finish and we'll return to waiting on callbacks. I'm not sure of any blocking restrictions. I just know it works. mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK rcpt to: 250 2.1.5 OK data 354 OK test . 451 4.4.0 Remote server not answering (connect) 250 2.0.0 CYKfHcsHZlBcCAAALhEySA Saved mail from: 250 2.1.0 OK rcpt to: 250 2.1.5 OK data 354 OK test . 250 2.0.0 EYKfHcsHZlBcCAAALhEySA Saved Jack From CMarcus at Media-Brokers.com Sun Sep 30 16:02:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 30 Sep 2012 09:02:03 -0400 Subject: [Dovecot] Log NAT IP address? Message-ID: <506842CB.8080501@Media-Brokers.com> Hi Timo/everyone, Currently we are logging the remote IP, but is there a way to show the IP address that the NAT connection is coming from? The reason I ask is, we are changing ISPs, and I would like to see in the logs when an external connection is coming from our OLD ISP connection, and when it is coming through our new one. We have a Watchguard firewall, and I have both External connections setup and working, and have just pointed our DNS records to the new public IP, and would like to be able to see which WAN connection/IP they are coming from. Thanks, -- Best regards, Charles From jbates at brightok.net Sun Sep 30 17:33:14 2012 From: jbates at brightok.net (Jack Bates) Date: Sun, 30 Sep 2012 09:33:14 -0500 Subject: [Dovecot] Log NAT IP address? In-Reply-To: <506842CB.8080501@Media-Brokers.com> References: <506842CB.8080501@Media-Brokers.com> Message-ID: <5068582A.6030507@brightok.net> On 9/30/2012 8:02 AM, Charles Marcus wrote: > Hi Timo/everyone, > > Currently we are logging the remote IP, but is there a way to show the > IP address that the NAT connection is coming from? > > The reason I ask is, we are changing ISPs, and I would like to see in > the logs when an external connection is coming from our OLD ISP > connection, and when it is coming through our new one. > > We have a Watchguard firewall, and I have both External connections > setup and working, and have just pointed our DNS records to the new > public IP, and would like to be able to see which WAN connection/IP > they are coming from. You could bind 2 internal IP Addresses to the server and have each NAT translation go to a different internal IP. Jack