April 2014 - dovecot - dovecot.org

Re: [Dovecot] integration of old mails into new dovecot Maildir
by K. C. Mohan - Personal 15 Apr '14

15 Apr '14

Thank you for your prompt reply and timely help. I will try to be very explanative in describing my situation. I will begin first with answering the questions you asked; 1) how are the messages storred on the old server, which storage backend do you use, maybe Maildir? Yes, Maildir. Dovecot version is 2.1.7. Virtual users are authenticated by mysql backend. Login is plain login. 2) the storage backend on the new server? Maildir. Dovecot Version 2.2.9. Mysql virtual users authentication. Plain login 3) do you care about the message UIDs? I Would like to preserve the UIDs as much as possible. However, the most important thing is for the users to have access to their old emails. 4) How many users do you have (approx.) and can you access the user's mailboxes via IMAP, e.g. using a master user or do you know their password or can you switch to a known password during migration of one particular user? I have about 50 total users. I can access their mailboxes via imap but I don't have their password handy. Mysql database with the userdata and passwords was imported into the new new server so, the users can get authenticated into the new server with the old server password. 5) Did you've changed namespaces? No. I did not change namespaces. - From your first post I figure that the basic mail server functionality of the new server is working, most importantly login, message delivery and retrieval. Yes, the new server is functional and all the users are able to log in, view delivered messages and retrieve the messages into their mail clients. However, they cannot view any old messages or retrieve any old messages. Here is the dovecot.conf of the old mail server; protocols = imap pop3 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%d/%n mail_privileged_group = mail first_valid_uid = 6060 ssl_cert= </etc/ssl/certs/dovecot.pem ssl_key =< /etc/ssl/private/dovecot.pem #ssl_ca = </etc/ssl/certs/gd_bundle.crt ssl = yes #ssl_cert = </etc/postfix/ssl/smtpd.crt #ssl_key = </etc/postfix/ssl/smtpd.key protocol imap { mail_plugins = autocreate quota imap_quota } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } protocol lda { mail_plugins = quota sieve quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r%n%nThe Subject of your message was: %s auth_socket_path = /var/run/dovecot/auth-master log_path = /var/log/dovecot-deliver.log postmaster_address = postmaster(a)sbm1.motherindia.com # sieve_path = /var/sieve/global.sieve } auth_mechanisms = plain login cram-md5 auth_debug = no auth_debug_passwords = yes auth_verbose = no auth_verbose_passwords = no disable_plaintext_auth = no instance_name = dovecot login_greeting = Dovecot ready. mail_debug = no passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } -----Original Message----- From: Steffen Kaiser [mailto:skdovecot@smail.inf.fh-brs.de] Sent: Monday, April 14, 2014 2:18 AM To: K. C. Mohan Cc: dovecot(a)dovecot.org Subject: Re: [Dovecot] integration of old mails into new dovecot Maildir -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 13 Apr 2014, K. C. Mohan wrote: Please, do not top post. > Thank you Steffen, for addressing my problem of migrating old mails > into new mail server. I went through the article you pointed to but > could not succueed. I will appreciate if you could guide me to step by > step process to migrate the mails. My old mail server used 2.x dovecot version. you post very little information: 1) how are the messages storred on the old server, which storage backend do you use, maybe Maildir? 2) the storage backend on the new server? 3) do you care about the message UIDs? 4) How many users do you have (approx.) and can you access the user's mailboxes via IMAP, e.g. using a master user or do you know their password or can you switch to a known password during migration of one particular user? 5) Did you've changed namespaces? - From your first post I figure that the basic mail server functionality of the new server is working, most importantly login, message delivery and retrieval. I have attached both the old and new servers configuration files as .txt attachments. > -----Original Message----- From: Steffen Kaiser > Sent: Thursday, April 10, 2014 6:14 AM > To: K. C. Mohan - Personal s > Cc: <mailto:dovecot@dovecot.org> dovecot(a)dovecot.org > Subject: Re: [Dovecot] integration of old mails into new dovecot > Maildir > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 8 Apr 2014, K. C. Mohan - Personal wrote: > >> I just installed dovecot 2.2.9 on to a new server and want to >> transfer all the old mails into the email accounts I imported from >> the previous server into MySQL database. I am not able to view the >> subfolders and or old mails that I copied into their respective >> maildir folders. I was very much appreciate if anyone could let me >> know how to import old mails into the new server. All I see, is the >> new mails that have come into the account since I started dovecot service in the new server. > > Check out <http://wiki2.dovecot.org/Migration> http://wiki2.dovecot.org/Migration . Read esp. about > "preserving messages' UIDs". > > If the old server uses Maildir, too, and you don't care about the > UIDs, copy the files over and you are done. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU0t9r3D1/YhP6VMHAQL21wgAx1GgDwQDWwDMPxbMQ+1vR3i7dsyUk5jm PNmRigoV8qylbn2AuZ1YMxGN2wEZfSl2zi/j2CtSEUX+d9LOf+lvI9V8zXkvSHT6 jYEHwuIQACjPUxWmh7x8XelvQvZHB9PEH56z8eJhkgyVxpae8Rbb1EHurvmxckef O0fdmCIglXJdJJ+B9WZ7JZVsB3M6PFFsllizS17+Hus6O2NeVaszLa7o7bk5XInD m+Co4e4sCVphVlssHRCIRa8Lj6nDC+D6g3RjCR+n0rVREXRbvEcSQPerEAqrZhf8 HB46x3V3n6C2fEu7RNWLB0+DzglHLBI5p9ZqAjJRMr3D9Xy9L5SKlQ== =IH25 -----END PGP SIGNATURE----- --- This email is free from viruses and malware because avast! Antivirus protection is active. <http://www.avast.com> http://www.avast.com --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com

3 2

[Dovecot] About the "nologin" extra field
by Axel Luttgens 14 Apr '14

14 Apr '14

Hello, While experimenting with the "nologin" extra field, I met a possibly overlooked behavior. Let's suppose the user database has those two columns: enabled: the user may/may not login (account active/not active) nologin: NULL if the mailbox is available, '!' if it is currently suspended (for maintenance reasons) The password_query: password_query = SELECT password, nologin, 'Maintenance' || nologin AS reason, [...] FROM [...] WHERE [...] AND enabled [...] Let's then experiment with pop connections. With: enabled set to false, nologin set to '!' the right password provided, following reply is emitted after an authentication failure delay: -ERR [AUTH] Authentication failed. So, as expected, the nologin value behaves as a "don't care" one. With: enabled set to true, nologin set to '!', the right password provided, following output is immediately emitted: -ERR [AUTH] Maintenance! Again, this is the kind of behavior I was expecting. With: enabled set to true, nologin set to '!', an incorrect password provided, there's the failure delay but the output still is: -ERR [AUTH] Maintenance! Here, I'm a bit dubitative... ;-) The delay seems to make the code's intent clear: to act as in the case of an authentication failure. But, notwithstanding the somewhat misleading explanation provided to a legitimate user with such a reply, isn't one unduly disclosing information about an account existence? Is this supposed to behave that way? If yes, what's the rationale? TIA, Axel

1 0

[Dovecot] POP3: Panic: Trying to allocate 0 bytes
by Urban Loesch 14 Apr '14

14 Apr '14

Hi, today I upgraded one of our dovecot servers from 2.117 to version 2.2.12 under Debian Squeeze. After the upgrade I got many of the following errors for pop3 users. My logfile shows: ... Apr 14 09:28:05 mailstore dovecot: pop3(user(a)domain.net pid:39688 session:<o1DTn/v2IABPNmv7>): Panic: Trying to allocate 0 bytes Apr 14 09:28:05 mailstore dovecot: pop3(user(a)domain.net pid:39688 session:<o1DTn/v2IABPNmv7>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x6bb0a) [0x7f1ae9a18b0a] -> /usr/lib/dovecot/libdovecot. so.0(+0x6bb86) [0x7f1ae9a18b86] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f1ae99d1e8f] -> /usr/lib/dovecot/libdovecot.so.0(+0x8148b) [0x7f1ae9a2e48b] -> dovecot/pop3() [0x4077f0] -> dovecot/pop3(client_command_execute+0x9d) [0x4 07d0d] -> dovecot/pop3(client_handle_input+0x80) [0x405810] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4e) [0x7f1ae9a28d2e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xaf) [0x7f1ae9a29e9f] -> /usr/lib/do vecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f1ae9a28db9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f1ae9a28e38] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1ae99d6c43] -> dovecot/pop3(main+0x2 57) [0x404a67] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f1ae9669c8d] -> dovecot/pop3() [0x4045b9] Apr 14 09:28:05 mailstore dovecot: pop3(user(a)domain.net pid:39688 session:<o1DTn/v2IABPNmv7>): Fatal: master: service(pop3): child 39688 killed with signal 6 (core dumps disabled) .... Have you any idea what the error could be? As this is a production server I switched back to version 2.1.17. Many thanks Urban Loesch doveconf -n: # 2.2.12 (978871ca81e7): /etc/dovecot/dovecot.conf # OS: Linux 3.4.67-vs2.3.3.9-rol-em64t-efigpt x86_64 Debian 6.0.9 ext4 auth_cache_negative_ttl = 0 auth_cache_size = 40 M auth_cache_ttl = 1 weeks auth_mechanisms = plain login auth_verbose = yes deliver_log_format = msgid=%m: %$ %p %w disable_plaintext_auth = no login_trusted_networks = $INTERNAL_IPS mail_gid = mailstore mail_location = mdbox:/home/vmail/%d/%n:INDEX=/home/dovecotindex/%d/%n mail_log_prefix = "%s(%u pid:%p session:<%{session}>): " mail_plugins = " quota mail_log notify zlib" mail_uid = mailstore managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql-account.conf driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from mail_log_group_events = no quota = dict:Storage used::file:%h/dovecot-quota quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_redirects = 10 zlib_save = gz zlib_save_level = 9 } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = mailstore mode = 0660 user = root } } service imap-login { inet_listener imap { port = 143 } process_limit = 256 process_min_avail = 25 service_count = 1 } service imap { process_limit = 256 process_min_avail = 25 service_count = 1 } service lmtp { inet_listener lmtp { address = * port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } process_limit = 256 process_min_avail = 25 service_count = 1 } service pop3 { process_limit = 256 process_min_avail = 25 service_count = 1 } service quota-warning { executable = script /usr/local/rol/dovecot/quota-warning.sh unix_listener quota-warning { user = mailstore } user = mailstore } ssl = no ssl_cert = </etc/dovecot/certs/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql-account.conf driver = sql } protocol lmtp { mail_fsync = optimized mail_plugins = " quota mail_log notify zlib sieve zlib" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep imap_id_log = * imap_logout_format = bytes=%i/%o session=<%{session}> mail_max_userip_connections = 40 mail_plugins = " quota mail_log notify zlib imap_quota imap_zlib" } protocol pop3 { mail_plugins = " quota mail_log notify zlib" pop3_logout_format = bytes_sent=%o top=%t/%p, retr=%r/%b, del=%d/%m, size=%s uidl_hash=%u session=<%{session}> pop3_reuse_xuidl = yes }

1 0

[Dovecot] Feature Request: handle mail connections of the same user from a same IP using just one mail process
by morrison 14 Apr '14

14 Apr '14

Hi, Dovecot Wiki states that mail process (IMAP/POP3) is able to handle multiple client connections as the same time to save CPU and memory resources (client_limit > 1). Although this approach is not recommended due to latency and probably security issues, it does help increase the overall capacity of a mail server. Is it possible (or planned) to add a feature such that mail client connections of the same user from a same IP are directed to a same mail process? In most cases, such connections come from a same mail client (e.g. in Thunderbird, each mailbox is a connection). The end user (human being) can work on one mailbox at a time, the latency will be non-obvious. And since all data being processed belong to the same user, security is a non-issue. Thanks,

3 2

Re: [Dovecot] integration of old mails into new dovecot Maildir
by K. C. Mohan 14 Apr '14

14 Apr '14

Thank you Steffen, for addressing my problem of migrating old mails into new mail server. I went through the article you pointed to but could not succueed. I will appreciate if you could guide me to step by step process to migrate the mails. My old mail server used 2.x dovecot version. K. C. Mohan -----Original Message----- From: Steffen Kaiser Sent: Thursday, April 10, 2014 6:14 AM To: K. C. Mohan - Personal s Cc: dovecot(a)dovecot.org Subject: Re: [Dovecot] integration of old mails into new dovecot Maildir -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 8 Apr 2014, K. C. Mohan - Personal wrote: > I just installed dovecot 2.2.9 on to a new server and want to transfer all > the old mails into the email accounts I imported from the previous server > into MySQL database. I am not able to view the subfolders and or old mails > that I copied into their respective maildir folders. I was very much > appreciate if anyone could let me know how to import old mails into the > new > server. All I see, is the new mails that have come into the account since > I > started dovecot service in the new server. Check out http://wiki2.dovecot.org/Migration . Read esp. about "preserving messages' UIDs". If the old server uses Maildir, too, and you don't care about the UIDs, copy the files over and you are done. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU0Zu/3D1/YhP6VMHAQKO0QgA7v5CowgoagbvEC3S0XI3k3QUHlsUczLE 38MedNzPw8cc69OHwzWxfxDiuCU2g2Q+Bv1vKMp+7aY2e9nCwUKcT0tOZGf7Nxrw IDG5dJbRtQSwTSOOvHSj1/SLY/tbYB71NkAPZl3XunFKcbkGl2BYKDfj+BuZ+WMZ pDj6B9S8GuV7BJNWlP4fRaHyrGDwfYuUmj1bTEoNuAFFPW11HfAwxRUQgnOKU9GJ zxERX2Tg7+Xj/RM49InQd8qqczYjsNte6M58a+6dwStGkPK4y0XD8nwsH8r4skvN utKKG+dRln+94oVnSG8WNNKpVmZt8K9oNHWq1CxVRIoqu2tUmppr0g== =qQnO -----END PGP SIGNATURE-----

2 1

[Dovecot] FTS autoindex username problem
by 李耀宗 13 Apr '14

13 Apr '14

I tried to set up Solr search server according to http://wiki2.dovecot.org/Plugins/FTS/Solr, and turned on autoindexing according to http://wiki2.dovecot.org/Plugins/FTS, but I found a strange behaviour that cause server search missing some emails. Here is the login information of my mail server: login name: user1 email: user1(a)example.com Now, when I turned on autoindexing, all autoindexed email are indexed using user1(a)example.com as username key, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1(a)example.com): Indexed 1 messages in INBOX But when I triggered on-demand indexing using IMAP server search, the emails are indexed using user1 as username key instead, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1): Indexed 1799 messages in INBOX Only the emails indexed using user1 can be searched, emails indexed using user1(a)example.com cannot be found. How can I configure autoindex to index email using user1 as username key? --- (Below is a follow-up question) In my mail server config, there are a few virtual mailbox users are using the following login information instead (different auth/password backend): login name: user2(a)virtmail.example.com email: user2(a)virtmail.example.com In this case, how can I distinguish user1(a)example.com and user2(a)virtmail.example.com, so that emails for user1(a)example.com can be indexed using user1 as username key, while user2(a)virtmail.example.com can be indexed using user2(a)virtmail.example.com as username key? .

2 2

[Dovecot] Still no messages from dovecot
by Steve Campbell 12 Apr '14

12 Apr '14

I'm still not seeing emails from the list. With help from others, I thought I had the IPs cleared since the change back in February, but apparently not. Can someone PM me and let me know what outgoing IP addresses the list is using please? steve campbell

2 1

[Dovecot] Will upgrade from from db4 to libdb break anything?
by A M 11 Apr '14

11 Apr '14

Hello, I just have a minor question. (Using Fedora 16) My current dovecot (2.2.10) was compiled with db4 (4.8.30). I am thinking of recompiling same dovecot (2.2.10) with libdb (5.2.36). I will be trying on live server so just wanted to know if this will this break any indexing or anything else? (Recompiling for Heartbleed, so might as well switch db version) AM.

1 0

[Dovecot] FTS autoindex username problem
by lee_yiu_chung＠yahoo.com 11 Apr '14

11 Apr '14

I tried to set up Solr search server according to http://wiki2.dovecot.org/Plugins/FTS/Solr, and turned on autoindexing according to http://wiki2.dovecot.org/Plugins/FTS, but I found a strange behaviour that cause server search missing some emails. Here is the login information of my mail server: version: 2.2.12 login name: user1 email: user1(a)example.com Now, when I turned on autoindexing, all autoindexed email are indexed using user1(a)example.com as username key, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1(a)example.com): Indexed 1 messages in INBOX But when I triggered on-demand indexing using IMAP server search, the emails are indexed using user1 as username key instead, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1): Indexed 1799 messages in INBOX Only the emails indexed using user1 can be searched, emails indexed using user1(a)example.com cannot be found. How can I configure autoindex to index email using user1 as username key? --- (Below is a follow-up question) In my mail server config, there are a few virtual mailbox users are using the following login information instead (different auth/password backend): login name: user2(a)virtmail.example.com email: user2(a)virtmail.example.com In this case, how can I distinguish user1(a)example.com and user2(a)virtmail.example.com, so that emails for user1(a)example.com can be indexed using user1 as username key, while user2(a)virtmail.example.com can be indexed using user2(a)virtmail.example.com as username key?

1 0

[Dovecot] FTS autoindex username problem
by lee_yiu_chung＠yahoo.com 11 Apr '14

11 Apr '14

I tried to set up Solr search server according to http://wiki2.dovecot.org/Plugins/FTS/Solr, and turned on autoindexing according to http://wiki2.dovecot.org/Plugins/FTS, but I found a strange behaviour that cause server search missing some emails. Here is the login information of my mail server: login name: user1 email: user1(a)example.com Now, when I turned on autoindexing, all autoindexed email are indexed using user1(a)example.com as username key, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1(a)example.com): Indexed 1 messages in INBOX But when I triggered on-demand indexing using IMAP server search, the emails are indexed using user1 as username key instead, as shown in maillog: dovecot: [ID 12345 mail.info] indexer-worker(user1): Indexed 1799 messages in INBOX Only the emails indexed using user1 can be searched, emails indexed using user1(a)example.com cannot be found. How can I configure autoindex to index email using user1 as username key? --- (Below is a follow-up question) In my mail server config, there are a few virtual mailbox users are using the following login information instead (different auth/password backend): login name: user2(a)virtmail.example.com email: user2(a)virtmail.example.com In this case, how can I distinguish user1(a)example.com and user2(a)virtmail.example.com, so that emails for user1(a)example.com can be indexed using user1 as username key, while user2(a)virtmail.example.com can be indexed using user2(a)virtmail.example.com as username key?

1 0