- dovecot - dovecot.org

Gmail to Dovecot migration
by Yuri 12 Aug '19

12 Aug '19

Hi everyone, I'm trying to migrate my gmail to dovecot following instructions at https://wiki.dovecot.org/Migration/Gmail Using -a 'virtual/All' option does not seem to work because the gmail folder with that name does not exist, but -a '[Gmail]/All Mail' seem to work. I'm using the following command: sudo doveadm \ -o imapc_host=imap.gmail.com \ -o imapc_user=user(a)gmail.com \ -o imapc_password=gmailpassword \ -o imapc_features="rfc822.size gmail-migration" \ -o imapc_port=993 \ -o imapc_ssl=imaps \ -o ssl_client_ca_dir=/etc/ssl/certs \ -o imapc_ssl_verify=yes \ backup -a '[Gmail]/All Mail' -O '-$GmailHaveLabels' -R -u user(a)exmaple.com imapc: The problem is this command only migrates approximately 25% of my emails by size. Google Takeout exports 4 times more data in mbox format, the size of exported mboxes aligns with what Google tells me a the bottom of gmail settings page. I experimented with other options of 'doveadm backup' command, but they all resulted in less data being migrated. The above options are the best I was able to achive so far. Another option I was considering is to convert Google Takeout mboxes to dovecot format. I tried using 'doveadm import' command, but got confused with various user and search query options of the command. The best I was able to achive was to output a folder under /Takeout which mimics my mbox structure, but with no imported messages. A third option I'm considering is to convert Google Takeout mboxes using mb2md command. Conversion seems to work fine, but the output is in the traditional (dotted) format, and for historical reasons I use 'LAYOUT=fs' option. So I need to do some extra work to convert folder names. And even if I do so, I'm still not sure if I can import Maildir to dovecot because dovecot's Maildir has other information, such as indices, logs, etc., not just email folders. I'll keep investigating the second and the third option because it seems that Google Takeout gives me all my emails, but I'm not sure if those are the right paths. I wanted to ask the experts about the best option to migrate gmail to dovecot. Has anyone being able to successfully migrate all their emails from gmail and what steps have been used for that? My Dovecot version: 2.3.4.1 Thank you, Yuri

1 1

What does `iterate_query` for SQL want as output?
by Coy Hile 10 Aug '19

10 Aug '19

Hi all, In an earlier thread, https://dovecot.org/pipermail/dovecot/2019-August/116694.html I got a lot of useful help about migration. On my older host, everything was static; on the newer host, I’m storing user information in Postgres. usernames are of the form <login@REALM>, say ‘hile(a)coyhile.xn--com-to0a as basically a Kerberos principal, and authentication and individual lookups work. My `users` table looks thus: mail=> \d users Table "public.users" Column | Type | Modifiers ----------+------+----------- username | text | not null domain | text | not null password | text | not null mail=> and contains, as an example: username | domain | password ------------------+-------------+------------------------------------------------------------------------------------------------------- hile(a)coyhile.com | coyhile.com | [REDACTED] (1 row) Naively, I’d expect something this to work for the iteration query: iterate_query = SELECT username, domain FROM users But, when I do that, I end up doveadm backup -D -A -R -f ssh -i id_rsa.dsync imap01.coyhile.com /opt/local/bin/doveadm dsync-server -A doveadm(hile@coyhile.com@coyhile.com): Info: User no longer exists, skipping [root@81716ec5-bca4-6d53-ed81-bd1a55d46b4f /tmp]# Note the extra “(a)coyhile.xn--com-9o0a in there. Thanks, — Coy Hile coy.hile(a)coyhile.com

2 6

Auth driver
by Riccardo Paolo Bestetti 09 Aug '19

09 Aug '19

Greetings! I'm planning to implement a new auth driver. It's going to be, in concept, similar to the Lua and CheckPassword drivers, in that it allows an user program to carry out the authentication and user enumeration steps. The rationale is that such a solution would provide better decoupling between Dovecot and the user authentication and enumeration logic. Existing integrated logic (auth/ directory in the source tree) is, even encompassing a great number of common use cases, ultimately inflexible. Writing custom drivers implementing logic, while being a solution to the inflexibility problem, is clearly not what drivers were engineered for: for one they don't support dynamic loading, making it necessary to recompile Dovecot to make changes, which is unacceptable in many situations, e.g. when using a packaged or containerized version of the software. In this email, I intend to explain what characteristics my driver will have and why it solves the proposed problem better than existing code (such as CheckPassword, Lua and proxy), in the hope to get useful feedback and I will pose some technical questions to which I could not find definitive answers by myself yet. These are going to be the characteristics of my driver (with particular emphasis to the differences to the existing drivers): - It is going to be a completely generic mechanism exposing all funcionality of Dovecot's interface, both password check and enumeration, and will fully support authentication methods with continuations. - It is going to be performant, communicating with external programs through sockets (unix or inet). - It is going to communicate using a simple, dynamic-schema, well-understood protocol, which is probably going to be JSON, being very easy to generate and parse. This is why the existing drivers do not already provide what I described: - CheckPassword is heavy (fork & 2*exec), and it's generally a mess, having initially been provided as a compatibility feature and having then been hacked to support additional functionality; it's architecturally unfeasable to support continuations with it and anyway any further extension will probably make it explode :) - Lua works but it's limited, in that it dictates a language and also dictates that the auth logic should run on the same server as Dovecot (and under Dovecot's control). It also doesn't seem to support plain password checks, but only user-password enumeration, making it non-generic (at least this is what can be inferred from the documentation and examples.) - Other possible solutions, such as (mis)using the proxy driver to communicate with an external program are not well documented and nothing guarantees the proxy protocol will not change anyway, being an internal feature. And now, the technical questions. I know these could be answered by reading the code, which is something I'm doing, but I'm finding some architectural details quite difficult to grasp due to their complexity: - From what I gather from [1], auth-worker processes only process a single request before being killed from auth. Would setting service_count to something different cause auth-workers to be reused or would it just leak the auth-workers? In the first case, is the preinit of the auth driver repeated, or is the existing instance of the auth driver reused? You could also point me to the relevant code sections which start, keep track of and initialize auth-workers. - How does a driver specify what to run in an auth-worker as opposed to in the auth process? (This is strongly related to the first question, and maybe it is more general, so you may want to answer this one first.) Please, feel free to only answer parts of my email. Any input on any of the matters is very welcome. Best regards, Riccardo P. Bestetti [1] https://wiki.dovecot.org/Services#auth-worker

2 1

Re: Upgrading to v2.3.X breaks ssl san?
by Aki Tuomi 09 Aug '19

09 Aug '19

> On 07/08/2019 14:28 telsch <telsch(a)gmx.de> wrote: > > > with v2.2.34 i can use: > > ssl_ca = </etc/ssl/ca-bundle.pem > ssl_cert = </etc/ssl-imap.pem > > after upgrade to v2.3.X it doesn't work like before. > > it's working if i manual cat ca-bundle.pem and ssl-imap.pem into one > file and using only: > > ssl_cert = </etc/ssl-imap.pem > > i thought ssl_ca is where to put the intermediate cert? (Sorry for duplicate mail, keyboard acted up...) No, that has always been a mistake and it was fixed in 2.3. Our SSL pages in documentation & wiki have always recommended concatenating the intermediates with the cert. Aki

3 3

IMAP folders showing no messages in Office 365 Outlook
by Andrew Bernard 08 Aug '19

08 Aug '19

I'm new to Dovecot. Using Dovecot IMAP with Postfix, everything works fine using Thunderbird. But with the same setup, no messages show in IMAP folders for user accounts created in Outlook, accessing exactly the same server as the accounts in Thunderbird. Dovecot version 2.3.7.1. Is there any configuration required that is special to support Office 365 Outlook clients? How to debug this? Andrew

3 3

Re: How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

Thank you so much, I tried it！ hfh(a)cndns.com 发件人： Aki Tuomi via dovecot 发送时间： 2019-08-08 15:47 收件人： hfh(a)cndns.com; dovecot 主题： Re: How to make the client tool only display emails within 30 days? On 8.8.2019 10.46, hfh--- via dovecot wrote: doveadm search -u matt mailbox "*" savedbefore 30d Dovecot's configuration or plugin can do similar effects like the above？ Outlook uses pop or imap to receive mail within 30 days. Virtual plugin can be used to do these kind of configurations. Aki

1 0

How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

doveadm search -u matt mailbox "*" savedbefore 30d Dovecot's configuration or plugin can do similar effects like the above？ Outlook uses pop or imap to receive mail within 30 days.

2 1

How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

How to make the client tool only display emails within 30 days? thanks！！！ hfh(a)cndns.com

2 1

Upgrading to v2.3.X breaks ssl san?
by telsch 07 Aug '19

07 Aug '19

3 5

Problem Solr and centos 7
by HTMLServices.it 07 Aug '19

07 Aug '19

Hi everyone Given that I am not an expert, I am doing tests with Solr, I installed following the guide but I have no benefits on the search, the search on the body on 28000 mails takes a few minutes and then goes to timeout. I had installed version 8.2.0 of solr then I thought that there is something not compatible with centos 7 so I installed version 7.7.0 as an example on the guide https://wiki2.dovecot.org/Plugins/FTS/Solr. I did several tests but I can't get it to work, this is the test server link: http://5.39.2.59:8987/solr/#/ I only changed the door because the 8983 is busy even making a query from the panel it seems to me not working .... thanks to all Bic

2 3