- dovecot - dovecot.org

Fwd: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15)
by Joan Moreau 14 Feb '19

14 Feb '19

Hi, THis is a core problem in Dovecot in my understanding. In my opinion, the rescan in dovecot should send to the FTS plugin the list of "supposedly" indexed emails (UID), and the plugin shall purge the redundant UID (i..e UID present in the index but not in the list sent by dovecot) and send back the list of UID not in its indexes to dovecot, so Dovect can send one by one the missing emails WHat do you think ? -------- Original Message -------- SUBJECT: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15) DATE: 2019-02-08 08:28 FROM: Leonard Lausen <notifications(a)github.com> TO: grosjo/fts-xapian <fts-xapian(a)noreply.github.com> CC: Subscribed <subscribed(a)noreply.github.com> REPLY-TO: grosjo/fts-xapian <reply+0022e607fd2eb3ff93959543198455bc7db5bdd58aa0286b92cf000000011874f1ae92a169ce185221c2(a)reply.github.com> doveadm fts rescan -A deletes all indices, ie. all folders and files in the xapian-indexes are deleted. However, according to man doveadm fts, the rescan command should only > Scan what mails exist in the full text search index and compare those to what > actually exist in mailboxes. This removes mails from the index that have already > been expunged and makes sure that the next doveadm index will index all the > missing mails (if any). Deleting all indices does not seem to be the intended action, especially as constructing the index anew may take very long on large mailboxes. -- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2]. Links: ------ [1] https://github.com/grosjo/fts-xapian/issues/15 [2] https://github.com/notifications/unsubscribe-auth/ACLmB9OB-7GaKIvhNc8sCgi7K…

2 2

Quota count and clone questions
by Robert Moskowitz 14 Feb '19

14 Feb '19

all this almost reads like I can drop maintaining the quota2 table? From https://wiki.dovecot.org/Quota/Count mailbox_list_index = yes # Avoid spending excessive time waiting for the quota calculation to finish when # mails' vsizes aren't already cached. If this many mails are opened, finish the # quota calculation on background in indexer-worker process. Mail deliveries will # be assumed to succeed, and explicit quota lookups will return internal error. mail_vsize_bg_after_count = 100 seems to belong in 10-mail.conf. That is where those var are shown. But: plugin { # 10MB quota limit quota = count:User quota quota_rule = *:storage=10M # This is required - it uses "virtual sizes" rather than "physical sizes" for quota counting: quota_vsizes = yes } I am having problems with. Right now for quota I have: plugin { quota = dict:user::proxy::sqlquota trash = /etc/dovecot/dovecot-trash.conf.ext } How do I reconcile these two? Then for clone: https://wiki.dovecot.org/Plugins/QuotaClone how does: mail_plugins = $mail_plugins quota quota_clone plugin { quota_clone_dict = redis:host=127.0.0.1:port=6379 } get replaced with something for mysql? dovecot-sql.conf.ext: driver = mysql connect = host=/var/lib/mysql/mysql.sock dbname=postfix user=postfix password=$Postfix_Database_Password default_pass_scheme = $cryptsha-CRYPT # following should all be on one line. password_query = SELECT username as user, password, concat('/home/vmail/', maildir) as userdb_home, concat('maildir:/home/vmail/', maildir) as userdb_mail, 101 as userdb_uid, 12 as userdb_gid FROM mailbox WHERE username = '%u' AND active = '1' # following should all be on one line user_query = SELECT concat('/home/vmail/', maildir) as home, concat('maildir:/home/vmail/', maildir) as mail, 101 AS uid, 12 AS gid, CONCAT('*:messages=30000:bytes=', quota) as quota_rule FROM mailbox WHERE username = '%u' AND active = '1' and dovecot-dict-sql.conf.ext: connect = host=/var/lib/mysql/mysql.sock dbname=postfix user=postfix password=$Postfix_Database_Password map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } thanks

1 1

unsubcribe
by Henry Otten 14 Feb '19

14 Feb '19

1 0

Archive maildir
by Gandalf Corvotempesta 14 Feb '19

14 Feb '19

Hi to all We have a maildir with about 180GB of emails. We have to archive them to a structure like: .Archive.YYY./MM.folder Are you aware of a script doing this ? I've found a perl script that doesn't spit in year and month and a very, very, very, very, very old python script that: 1) doesn't manage base64 encoded subject properly 2) doesn't work with python 3.x (that is able to manage base64 encoded subject properly Any idea ?

6 9

Maintaining table quota2
by Robert Moskowitz 14 Feb '19

14 Feb '19

Does dovecot compute that values for quota2 with each email it updates for the user? Or only an incremental change? I ask because I am looking at migrating all the user mail from the old server to the new and building a new sql database. All I see is: dovecot-dict-sql.conf.ext: connect = host=/var/lib/mysql/mysql.sock dbname=postfix user=postfix password=$Postfix_Database_Password map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } what is actually done with this table? thanks

3 11

problem witht working director
by Maciej Milaszewski IQ PL 13 Feb '19

13 Feb '19

Hi I have dovecot director with 2-3 ring and 4 dovecot My ring is : 10.0.100.2 9090 right never synced 8 37305225 39763861 0 723 2019-02-13 16:02:04 2019-02-13 16:02:04 10.0.100.3 9090 left never synced 8 57409664 8707830 0 737 2019-02-13 16:02:04 2019-02-13 16:02:04 10.0.100.4 9090 self never ring synced 1 - - - - - - 10.0.100.4 - This is my default server where clients connect - default direcor 2.2.36 10.0.100.3 - second direcot 2.2.36 10.0.100.2 - second director with older ver 2.2.18 Today I found the error in my default mailserver Feb 13 13:21:23 kernel: [24253349.641695] qmail-remote: page allocation failure: order:2, mode:0x204020 and problem with logging via imap and pop3 I try "doveadm director status" i get time out I try restart dovecot and get: Feb 13 13:21:03 thebe3 dovecot: master: Fatal: Dovecot is already running? Socket already exists: /var/run/dovecot/login/director before failure i found: "imap-login: Error: write(proxy-notify) failed: Resource temporarily unavailable" but my setting they seem correct https://paste.debian.net/1067596/ I moved IP (via keepalive) all to other director (secound servcer) And when I try "doveadm director status" i get time out https://paste.debian.net/1067601/ I do not understand why I had a problem with second server and restart dovecot in server2 restart solved the problem ?

1 0

smtp_address_cmp does not ignore case of localpart
by Andreas Jobs 12 Feb '19

12 Feb '19

Hi, in src/lib-smtp/smtp-address.c line 759 the localpart is compared with null_strcmp, which does not ignore the case. While this is the correct behavior (according to the RFC) it causes problems in real life. In my case the vacation notices are not sent, if someone writes to First.Last(a)ruhr-uni-bochum.de because :addresses contains (only) first.last(a)ruhr-uni-bochum.de. Regards, Andreas -- ! AJ3630-RIPE @ RUB8-RIPE Network Operation Center ! ! Dezernat 5.I Ruhr-Universitaet Bochum ! ! The amount of energy necessary to refute bullshit is ! ! an order of magnitude bigger than to produce it. -- Alberto Brandolini !

2 3

Can't sync mailbox XXX: Mailbox isn't selectable
by Tamas Hegedus 12 Feb '19

12 Feb '19

Hi, I have users with file system folders, which contain mbox files and dovecot indexes, in the ~/mail directory. If I run dsync, I have an error message and the sync can not be completed: Can't sync mailbox FOLDER_NAME: Mailbox isn't selectable I could not find any solution via google or any hint what to do in this situation. Thanks for your help and suggestion, Tamas -- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org --- This email has been checked for viruses by AVG. https://www.avg.com

2 1

doveadm import with subfolder oddity
by Mark Moseley 12 Feb '19

12 Feb '19

This has got to be something weird in my config. And the standard disclaimer of '"happy to post doveconf -n, but wanted to see if this is normal first" :) Background: Ubuntu Xenial, running 2.2.36. Mailbox type is mdbox and I've got a period separator in my inbox namespace: namespace { hidden = no inbox = yes list = yes location = mailbox Spam { auto = no autoexpunge = 1 weeks special_use = \Junk } mailbox Trash { auto = no special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } If I do a import for a regular folder under INBOX, it works just fine: doveadm import -u testbox2(a)testing.local -U testbox1(a)testing.local mdbox:~/mdbox INBOX all mailbox Sent ... returns happily, message count gets incremented If I try to do the same with a subfolder (and a subfolder that most definitely exists on both source and destination side), I get an error: doveadm import -u testbox2(a)testing.local -U testbox1(a)testing.local mdbox:~/mdbox INBOX all mailbox Sub.Sub1 doveadm(testbox2(a)testing.local): Error: remote(10.1.17.98:4000): Mailbox Sub.Sub1: Mailbox sync failed: Mailbox doesn't exist: Sub.Sub1 If I use / instead of . in my query, it works: doveadm import -u testbox2(a)testing.local -U testbox1(a)testing.local mdbox:~/mdbox INBOX all mailbox Sub/Sub1 ... returns happily and message count gets incremented. Since we're using '.' as our separator, that was a bit unexpected :) Ironically, if I'm doing a IMAPc 'import', it works just fine with a query of 'all mailbox Sub.Sub1'. It's only when importing from a local src and local dest (i.e. source_location == mdbox:~/mdbox) that it fails. With source_location set to 'imapc:', it works. I imagine that's due to using straight IMAP on the source side. Likely a misconfig on my part? Expected behavior? I can see in the strace that the error is triggered when doveadm is looking at the source mailbox. It looks for mdbox/mailboxes/Sub.Sub1/dbox-Mails first, then falls back to mdbox/mailboxes/Sub/Sub1/dbox-Mails (which it finds). Then a little bit later in the strace, it again looks for mdbox/mailboxes/Sub.Sub1/dbox-Mails (which it doesn't find) but doesn't try mdbox/mailboxes/Sub/Sub1/dbox-Mails this time, and then spits out 'Mailbox Sub.Sub1: Mailbox sync failed: Mailbox doesn't exist: Sub.Sub1'. With a query of 'all mailbox Sub/Sub1', the stat() is for mdbox/mailboxes/Sub/Sub1/dbox-Mails which it finds and uses happily. Having to substitute the '.'s for '/'s in the 'mailbox' part of the query isn't an awful workaround, but it very much feels like I'm doing something wrong. This is a production setup, so everything else is otherwise working fine. But I've only just begun working with 'doveadm import', so I might be turning up some issues with my config. Thanks! Sorry I'm so verbose :)

1 1

Doveadm service as non-root user
by Mark Moseley 12 Feb '19

12 Feb '19

Running: Ubuntu xenial, dovecot 2.2.36 I've been working on moving our user base from maildir to mdbox and trying to come up with solutions for things like moving emails around. In the past, with maildir, our support guys could just mv the files around and done. For mdbox, I've been working on getting things set up to use doveadm. One weirdness I've seen is that in imports (i.e. doveadm import), mail gets copied correctly but the resulting files are left with root ownership (I don't have 'service doveadm' 'user' set, so I guess it defaults to root). It's typically new m.* files as well as the dovecot.list.index and dovecot.list.index.log files. Looking at strace, no chown is done on them, nor was there setuid. The import had no trouble finding the correct user in the db, so I know that it knows the correct UID (I can see it just fine in debug logs too). And it will happily import to existing m.* files with no permissions issues (but considering it's running as root, I wouldn't expect it to). I've seen this using 'import' via IMAPc as well as with both src and dest on the same server. I can see this behavior in both scenarios. We have a single shared UID for mail, so especially in that "src/dest on same server" case, it's not a matter of UID-mismatch. It's a director setup, so all doveadm commands are coming through the director. If I run the import directly on the backend (which obviously would be a bad idea in real life), the ownership of new m.* files seems to be correct (I can see it setuid'ing to the correct UID from userdb in strace). If I run the import on the director, I can get a new root-owned file every time it rolls over to the next m.* file. Two questions: * Is that a bug? Is this expected behavior? Seems like the expected thing would be to use the UID from userdb and either do a setuid (just like running 'doveadm import' locally did) or chown'ing any new files to the correct UID. I always always assume misconfiguration (vs bug, since it's almost never a bug) but I'm baffled on this one. * I see that it's possible to set a user for service doveadm and the wiki even suggests that it's a good idea in a single UID setup. If there are no mailboxes with any other UIDs, *will setting 'service doveadm' to the same UID possibly break anything*? I can't think of why it would, but I want to be duly diligent. Plus I'm a little leery about closing the door to ever having additional UIDs for mailboxes. Happy to provide 'doveconf -n' but wanted to check first, before spending 15 minutes gently obfuscating it :)

2 3