- dovecot - dovecot.org

Re: fts_solr: Error: fts_solr: received invalid uid '0'
by Fabian Kuran 10 Dec '19

10 Dec '19

Hi, > Am 23.09.2019 um 09:36 schrieb Filip Hanes <filip.hanes(a)ringier.sk>: > > > št 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot(a)dovecot.org <mailto:dovecot@dovecot.org>> napísal(a): > Are there any other hints or tips regarding this „invalid uid ‚0‘"-message? > from fts-solr source: > case SOLR_XML_CONTENT_STATE_UID: > if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) { > i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len)); > It means that plugin tries to parse '0' when in state of parsing uid in xml response from solr or there is a bug in xml response parser. > I would enable rawlog_dir in config and look there for unusual response or extra tags near usual uid in xml response. For example: > plugin { > fts = solr > fts_solr = url=... rawlog_dir=/var/log/fts-solr/ > } > Thanks for this tip! The rawlog_dir option is not yet supported in our version 2.3.4. What we could provide now would be the XML-Response from Solr. The corresponding search query sometimes causes this error. But Solr's answer seems to always remain the same - in my opinion this already indicates a bug of Dovecot? The XML response can be downloaded here: https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr Kind regards, Fabian

4 6

Re: Dovecot & OAuth
by Aki Tuomi 10 Dec '19

10 Dec '19

Hi! It seems there is a bug in the oauth2 driver, it loads the cert files wrong way. I'll make an internal bug report of this. Aki > On 06/12/2019 16:42 mizuki <mizuki0621(a)gmail.com> wrote: > > > Hi, > > For troubleshooting purposes, I change the read/write permissions on the certs and confirmed 'dovecot' can read them w/o problem, but still seeing the same errors. :( > Mizuki > > > On Fri, Dec 6, 2019 at 1:35 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > > > Is the key/cert pair readable by dovecot user? auth process does not run as root. > > > > > > > > > > You can add > > > > > > > > > > service auth { > > > > extra_groups = ssl_cert > > > > } > > > > > > > > > > and chgrp the cert to ssl_cert to allow access to the cert. > > > > > > > > > > Aki > > > > > On 06/12/2019 04:16 mizuki via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > I changed some of the tls options following the document, now config is following: > > > > > > > > > > > > > > > > > > > > > > > > tokeninfo_url = https://keycloak.com/auth/realms/mail/protocol/openid-connect/token > > > introspection_url = https://dovecot:7598e21b-ec34-481f-80d0-059bddae0923@keycloak.com/auth/real… > > > introspection_mode = post > > > debug = yes > > > rawlog_dir = /tmp/oauth2 > > > #force_introspection = yes > > > username_attribute = username > > > #active_attribute = active > > > #active_value = true > > > tls_ca_cert_file = /etc/pki/CA/certs/incommon-rsa-server-ca.crt > > > tls_cert_file = /etc/pki/dovecot/certs/dovecot.pem > > > tls_key_file = /etc/pki/dovecot/private/dovecot.pem > > > > > > > > > --------------- > > > > > > > > > > > > > > > The debug log is showing now slightly different msg ex: > > > > > > Dec 5 21:09:59 mktst4 dovecot: auth: Error: oauth2(mizuki,10.0.2.1,<29b4iv+YKuuCx5Tr>): oauth2 failed: Couldn't initialize SSL context: Can't load SSL certificate: There is no valid PEM certificate. > > > > > > > > > > > > > > > Still not able to connect to the keyclaok server. :( > > > > > > > > > > > > > > > > > > PS: Dovecot & Keycloak severs are both using the same legit cert/key pair with CA file configured. > > > > > > > > > > > > > > > Thanks! > > > > > > Mizuki > > > > > > > > > > > > > > > > > > > > > On Thu, Dec 5, 2019 at 3:06 PM Aki Tuomi < aki.tuomi(a)open-xchange.com> wrote: > > > > > > > > > > Before declaring it not ready for prime time, did you try setting > > > > > > > > tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt > > > > > > > > In the oauth2 configuration file as documented in https://doc.dovecot.org/configuration_manual/authentication/oauth2 ? > > > > > > > > Aki > > > > > > > > > On 05/12/2019 21:58 mizuki via dovecot < dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > We'd like to enable OAuth with Keycloak in Dovecot, after enabling 'OAUTHBEARER XOAUTH2' in Dovecot based on online document, I can confirm Dovecot is ready for OAuth using openssl command, however when the auth request comes in, it failed in establishing a SSL connection with Keycloak server on port 443, shown as following in debug logs. I can confirming using commands 'openssl s_client -connect <keycloak_server>:443' or 'curl -v https://<keycloak_server/' all returns normal and no errors. Altering some of the SSL options in dovecot such as 'ssl_ca = </etc/pki/CA/certs/root_ca.pem' or 'ssl_client_ca_file = </etc/pki/CA/certs/root_ca.pem' does not help either. The certificate are NOT self-signed but signed the legit authorities. So I'm not sure why dovecot could not establish the connections. > > > > > > > > >

3 4

OAuth2 mail client
by mizuki 10 Dec '19

10 Dec '19

Hi all, I'm wondering if there are any IMAP client software alternative to Thunderbird who can handle OAuth2 other than using gmail, yahoo etc (ex, talk to local auth provider)? Thunderbird does not seem to support well at the time being so I'm wondering what other choices we may have for our user communities. Thank you very much. Mizuki

1 0

Problem With Quota Dovecot
by Gter Marcelo 09 Dec '19

09 Dec '19

Hi Dears, I have one problem with Dovecot in My mailserver. Dovecot Quota setting is not working fine. plugin { quota = dict:User quota::file:MAILDIR/dovecot-quota quota_rule = *:storage=1GB } For example if i put the limit in 1GB, i have accounts in my mailbox with size of maildir in 3gb,4gb for exemple and if i running the command doveadm quota get -u ACCOUNT_MY_USER The user with 2GB should be return 100% or some number bigger , but return 45% used only, If i force running one force-resync, the quota-get return the size correct showing more than 100% Someone can Help me ?

2 4

[whishlist] new option for 'doveadm purge'
by Michael Grimm 09 Dec '19

09 Dec '19

Hi, I do store mail in mdbox format of 150m in size (dovecot 2.3.9). Once in a while I do experience mdbox files of smaller size, even after applying 'doveadm purge' and previous expunges by the users. like: -rw------- 1 vmail dovecot 104854595 Feb 9 2019 /var/mail/.maildirs/userX/storage/m.22 -rw------- 1 vmail dovecot 29088478 Mar 8 2019 /var/mail/.maildirs/userX/storage/m.31 -rw------- 1 vmail dovecot 98210890 Mar 20 2019 /var/mail/.maildirs/userX/storage/m.39 (Currently the counter is at file number 129.) Well, I never experienced missing mail or alike, but these "holes" in filesize irritates me, and yes, it is more or less a cosmetic issue. Nevertheless, I do sometimes want to get rid of these "holes" by backing up all mail and re-injecting the backup into a vanilla account of that user. And I used this approach when I wanted to store all mail messages in larger mdbox files; again, rather a cosmetic issue. BUT that takes a very, very long time contrary to the speed of 'doveadm purge'. Unfortunately, that command starts somewhere with more recent mdbox files and never from scratch (oldest mdbox file). Whishlist: Would it be much of an effort to implement an option like: 'doveadm purge -f' and '-f' standing for 'force' or 'from scratch' or 'from the very first message found' or 'you name it'? Thanks in advance and thanks for Dovecot and with kind regards, Michael

2 2

bash script hook lda_mailbox_autocreate for generate mail-crypt user encrypted private key with user password
by uxqex4efpu＠elude.in 09 Dec '19

09 Dec '19

What it is way most best for causing bash script run (as root) of time mailbox created (lda_mailbox_autocreate)? I use dovecot 2.3.4.1 in Debian 10. And I use of mail-crypt-plugin https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ I setup mail-crypt for requiring user encrypted EC key (mail_crypt_require_encrypted_user_key = yes). I want for passphrase encrypt EC key using client plaintext password. There is credential no stored on server. But for user with use password too bad, I concatenate user plaintext password with random salt. And then string to SHA512() hash and use as decryption key (mail_crypt_private_password) for EC private key. For above I have plugin config > mail_plugins = $mail_plugins mail_crypt > plugin { > mail_crypt_curve = secp256k1 > mail_crypt_require_encrypted_user_key = yes > mail_crypt_save_version = 2 > } And for returning userdb_mail_crypt_private_password, I have sql query > password_query = SELECT username, password, \ > SHA2( CONCAT('%w',salt), 512 ) AS userdb_mail_crypt_private_password \ > FROM virtual_users WHERE username='%u'; But how I generate key of user automatically? Note for generating key of user, I need user password plaintext. I never save plaintext password of user of the server. Also user of note creates in PHP of web of the server. And for security I do not allow PHP exec shell (php.ini disabled_functions). Definitely not leaving PHP doveadm access! For solving subject to generate user key encrypted, I do imap of call of the service 'imap-postlogin' the service likes document "Post-login scripting' write https://doc.dovecot.org/admin_manual/post_login_scripting/ And 'imap-postlogin' execute my custom script with 'script-login' binary https://github.com/dovecot/core/blob/8606e1abb90a1c91357b84bf547a89564d0535… Here it is config for above > service imap { > executable = imap imap-postlogin > } > service imap-postlogin { > executable = script-login /usr/local/bin/generateKeys.sh > unix_listener imap-postlogin { > } > } And generateKeys.sh it is script simple for generating keys with sha256() hash product mysql. Variable of note ${MAIL_CRYPT_PRIVATE_PASSWORD} automatically put of 'userdb_mail_crypt_private_password' return of mysql field of query when documented https://doc.dovecot.org/admin_manual/post_login_scripting/running-surroundi… > Fields returned by userdb lookup with their keys uppercased > (e.g. if userdb returned home, it's stored in HOME). Here generatekeys.sh > #!/bin/bash > if [ `/usr/bin/doveadm mailbox cryptokey list -u "${USER}" -U > /dev/null | wc -l` -lt 2 ]; then > /usr/bin/doveadm -o "plugin/mail_crypt_private_password=${MAIL_CRYPT_PRIVATE_PASSWORD}" mailbox cryptokey generate -u "${USER}" -U > /dev/null > fi > exec "$@" This work! But I want more good. By why execute each login? Possible has generateKeys.sh execute in the times only of dovecot create mailbox (lda_mailbox_autocreate) instead?

2 4

Execute a script from dovecot when authentication fails
by nb 09 Dec '19

09 Dec '19

Hello, My name is Noury, and I’m new here. I’ve been using dovecot IMAP for years, but I want to do now some extra things. I want to call a script from within dovecot, to build a suspicious IP addresses list when authentication fails. I have tried the PostLoginScripting technic. It works fine but it doesn’t suit my needs. The script executes only after a successful login. I want to execute my script only when auth fails. If I can execute a script and check into it if auth failed, I would be nice too. A kind of « PreLoginScript" Is there a way to achieve this? I’m reading now auth.log file, but I would like to have a realtime processing. Thanks in advance. Noury

2 1

Re: How do I set all my mailboxes to "subscribed"
by Aki Tuomi 09 Dec '19

09 Dec '19

2 1

Shared namespace is empty after migration to new Dovecot installation
by Peter Reinhold 09 Dec '19

09 Dec '19

Hi all, I've used most of this weekend finalizing my migration from an old server running Dovecot 2.2.22 to a brand new installation running 2.3.8. In general, the migration was painless, everything just clicked, but checking up on the finer details post migration i've found out that user shared folders does not seem to work anymore, and I simply can't get it to work. The setups are more or less identical, biggest change is probably the addition of a Solr search index, and i've done very little changes in the configuration files. The acl_shared_dict file is being created and updated when I share a folder In the user thats sharing the folder, the dovecot-acl and dovecot-acl-list files are created and look the same. But, the user that should be reading the shared folder can not see it, the Shared namespace is simply empty, which it is not on the old installation, and namespace setup is identical across the two servers. (Except for the CaSiNg of the location of the index and indexpvt) Logging in and doing an IMAP command : A1 LIST "Shared/" "*" yields this on the old server. (I've replaced the sharing users e-mail to user(a)sharer.dk and the reading users to user(a)reader.dk) A1 LIST "Shared/" "*" * LIST (\HasNoChildren) "/" Shared/user(a)sharer.dk A1 OK List completed (0.000 + 0.000 secs). and this on the new A1 LIST "Shared/" "*" A1 OK List completed (0.006 + 0.000 + 0.005 secs). Log output from the new server below (i've only included the events happening during the LIST account) -- New, non-working log start -- Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:/data/mail/%d/%n/maildir:INDEX=~/maildir/shared/%u:INDEXPVT:~/maildir/shared/%u shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= fts: Indexes disabled for namespace 'Shared/%u/' acl: initializing backend with data: vfile acl: acl username = user(a)reader.dk acl: owner = 0 acl vfile: Global ACLs disabled acl vfile: file /data/mail/reader.dk/user/maildir/dovecot-acl not found acl vfile: file /data/mail/reader.dk/user/maildir/.Junk/dovecot-acl not found acl vfile: file /data/mail/reader.dk/user/maildir/.Trash/dovecot-acl not found acl vfile: file /data/mail/reader.dk/user/maildir/.Sent/dovecot-acl not found auth-master: userdb lookup(user(a)sharer.dk): Started userdb lookup auth-master: conn unix:/var/run/dovecot/auth-userdb: Connecting auth-master: conn unix:/var/run/dovecot/auth-userdb (pid=1636,uid=0): Client connected (fd=16) auth-master: userdb lookup(user(a)sharer.dk): auth USER input: user(a)sharer.dk uid=120 gid=129 home=/data/mail/sharer.dk/user auth-master: userdb lookup(user(a)sharer.dk): Finished userdb lookup (username=user(a)sharer.dk uid=120 gid=129 home=/data/mail/sharer.dk/user) Namespace Shared/: Using permissions from : mode=0700 gid=default -- New, non-working log end -- and here from the old, working installation -- Old, working log start -- Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:/data/mail/%d/%n/maildir:INDEX=~/Maildir/shared/%u:INDEXPVT=~/Maildir/shared/%u shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= acl: initializing backend with data: vfile acl: acl username = user(a)reader.dk acl: owner = 0 acl vfile: Global ACLs disabled auth input: user(a)sharer.dk uid=118 gid=126 home=/data/mail/sharer.dk/user namespace/default/inbox=yes maildir++: root=/data/mail/sharer.dk/user/maildir, index=/data/mail/sharer.dk/user/Maildir/shared/user(a)sharer.dk, indexpvt=/data/mail/sharer.dk/user/Maildir/shared/user(a)sharer.dk, control=, inbox=/data/mail/sharer.dk/user/maildir, alt= acl: initializing backend with data: vfile acl: acl username = user(a)sharer.dk acl: owner = 1 acl vfile: Global ACLs disabled maildir++: root=/data/mail/sharer.dk/user/maildir, index=/data/mail/reader.dk/user/Maildir/shared/user(a)sharer.dk, indexpvt=/data/mail/reader.dk/user/Maildir/shared/user(a)sharer.dk, control=, inbox=/data/mail/sharer.dk/user/maildir, alt= acl: initializing backend with data: vfile acl: acl username = user(a)reader.dk acl: owner = 0 acl vfile: Global ACLs disabled acl vfile: reading file /data/mail/sharer.dk/user/maildir/dovecot-acl acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Junk acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Trash acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Sent acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Drafts acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Junk acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Trash acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Sent acl: Mailbox not in dovecot-acl-list: Shared/user(a)sharer.dk/Drafts -- Old, working log end -- The two logs are quite different, the new one seems to do user lookups via the auth socket, which the old one does not, and then just stops there. I'm not going to try and speculate more, I hope that someone here can see something from the logs, as I simply do not know how to fix this problem. -- Peter Reinhold

2 2

director option
by soumitri＠iitk.ac.in 09 Dec '19

09 Dec '19

Hi, I think option-2 below seems better or am I missing something with director concept. 1. As director and back-end in same server (broken) not recommended, I need minimum 4 servers (2 for director, 2 back-end IMAP). Indexes and mail must be in NFS shared storage(not optimal), as a user may land on any of the back-end servers (assuming no static mapping of user from pass DB). 2) Proxy_may in passDB (static mapping of users to servers makes balancing little partial but can be admin optimized) can achieve the above setup with 2 servers without director. Here indexes can be on their local disk (efficient) and mail on shared storage (NFS). Both case I can use RR DNS to get users IMAP traffic to front end. Any guidance on this is appreciated. -- Sincerely, Soumitri Mishra

1 1