- dovecot - dovecot.org

[Dovecot] Upload / migrate old mails: iPhone shows oldest mails first
by Denny Schierz 19 Dec '12

19 Dec '12

hi, I've got a very strange behavior: I uploaded old mails from a local folder (thunderbird / osx) to my Dovecot (Debian Squeeze) and now, the iphone (ios6) show the oldest mails first. The only thing I can imagine is, that Dovecot got from Thunderbird the newest mails first and last the old mails, so that in the last mdbox are the old one. So, if it is correct, how can I fix it? cu denny

2 1

[Dovecot] Path for fixing sequence lookup of newly created mails in transaction view
by Potapov Sergey 19 Dec '12

19 Dec '12

Hi! It seem that tview_lookup_seq_range works incorret, if I request uids from 3 to 3, and I have new uids 1,2,4,5 it will return seq for uid=4. Fixing patch is included as attach. ------ Sergey Potapov

2 1

Re: [Dovecot] dovecot-lda (2.1.12) segfaults
by Joseph Tam 19 Dec '12

19 Dec '12

Timo Sirainen writes: > Can you still reproduce this in any way? Yes, I have 6 sets of user INBOX index caches that will crash dovecot-lda. The actual content of the INBOX is irrelevant (crash probably happens before INBOX is opened). > I found two bugs, would be nice to know if they solve it: > > http://hg.dovecot.org/dovecot-2.1/rev/2f848393f78e > http://hg.dovecot.org/dovecot-2.1/rev/bded819417d9 No, these patch don't help. It crashes in the same place with the same value of field_hdr. Here's the full backtrace: #0 0xff2a0474 in mail_cache_header_fields_read (cache=0x5c250) at mail-cache-fields.c:325 field_hdr = (const struct mail_cache_header_fields *) 0x20 field = {name = 0x402 <Address 0x402 out of bounds>, idx = 4282351288, type = MAIL_CACHE_FIELD_VARIABLE_SIZE, field_size = 4282335628, decision = MAIL_CACHE_DECISION_TEMP, last_used = -14558816} last_used = (const uint32_t *) 0x64584 sizes = (const uint32_t *) 0xc types = (const uint8_t *) 0x64888 "" decisions = (const uint8_t *) 0x64900 "" p = 0x24a38 "�\035\212@��" names = 0x0 end = 0x64a50 "" orig_key = (void *) 0xffbfee38 orig_value = (void *) 0x64550 fidx = 411784 new_fields_count = 4280126016 dec = MAIL_CACHE_DECISION_NO max_drop_time = 376804 offset = 32 i = 4282348464 #1 0xff29e8cc in mail_cache_compress_locked (cache=0x5c250, trans=0x645e0, unlock=0xffbfeeef) at mail-cache-compress.c:361 dotlock = (struct dotlock *) 0x2ea00 st = {st_dev = 235718347, st_pad1 = {874, 0, 0}, st_ino = 0, st_mode = 0, st_nlink = 0, st_uid = 0, st_gid = 0, st_rdev = 3720, st_pad2 = {0, 0}, st_size = 3720, st_atim = { tv_sec = 410816, tv_nsec = -12631336}, st_mtim = {tv_sec = 514, tv_nsec = -12631336}, st_ctim = {tv_sec = 65536, tv_nsec = 0}, st_blksize = 0, st_blocks = 1621028016851520, st_fstype = "\000\000\000\004\000\000\000\003\212\000\000\000\000\005�P", st_pad4 = {-4198784, -14028952, 39394339, 377424, 0, 16777216, 3, 4}} old_mask = 4282348464 file_seq = 4 old_offset = 4290768372 ext_offsets = {arr = {buffer = 0xffbfee10, element_size = 4280930288}, v = 0xffbfee10, v_modifiable = 0xffbfee10} offsets = (const uint32_t *) 0x0 data = (const void *) 0xff3f4380 i = 0 count = 1 fd = 0 ret = 377424 #2 0xff29efe0 in mail_cache_compress (cache=0x5c250, trans=0x645e0) at mail-cache-compress.c:489 unlock = false ret = 411764 __FUNCTION__ = "mail_cache_compress" #3 0xff2a3e28 in mail_cache_transaction_compress (ctx=0x5e3b8) at mail-cache-transaction.c:180 cache = (struct mail_cache *) 0x5c250 view = (struct mail_index_view *) 0x644c0 trans = (struct mail_index_transaction *) 0x645e0 ret = 2424 #4 0xff2a40b8 in mail_cache_transaction_open_if_needed (ctx=0x5e3b8) at mail-cache-transaction.c:241 cache = (struct mail_cache *) 0x5c250 ext = (const struct mail_index_ext *) 0x1e idx = 154968 i = 1 __FUNCTION__ = "mail_cache_transaction_open_if_needed" #5 0xff2a6e94 in mail_cache_field_want_add (ctx=0x5e3b8, seq=1, field_idx=12) at mail-cache-transaction.c:1048 decision = 153968 #6 0xff27e8e8 in index_mail_parse_header_register_all_wanted (mail=0x5efa8) at index-mail-headers.c:175 _mail = (struct mail *) 0x5efa8 all_cache_fields = (const struct mail_cache_field *) 0x25970 i = 12 count = 26 #7 0xff27ec90 in index_mail_parse_header_init (mail=0x5efa8, headers=0x0) at index-mail-headers.c:230 _data_stack_cur_id = 2 data = (struct index_mail_data *) 0x5f058 match = (const uint8_t *) 0x641a0 "" i = 0 field_idx = 4290769328 match_count = 2155905152 __FUNCTION__ = "index_mail_parse_header_init" #8 0xff27f5c8 in index_mail_cache_parse_init (_mail=0x5efa8, input=0x64058) at index-mail-headers.c:376 mail = (struct index_mail *) 0x5efa8 input2 = (struct istream *) 0x641a0 __FUNCTION__ = "index_mail_cache_parse_init" #9 0xff2299cc in mbox_save_get_input_stream (ctx=0x5e6e0, input=0x637c8) at mbox-save.c:411 filter = (struct istream *) 0x0 ret = (struct istream *) 0x5edd0 cache_input = (struct istream *) 0x25990 streams = {0x20202020, 0x2e938, 0xa202020} #10 0xff22a084 in mbox_save_begin (_ctx=0x5e6e0, input=0x637c8) at mbox-save.c:520 ctx = (struct mbox_save_context *) 0x5e6e0 t = (struct mbox_transaction_context *) 0x5de88 save_flags = MAIL_RECENT offset = 0 __FUNCTION__ = "mbox_save_begin" #11 0xff24e9c0 in mailbox_save_begin (ctx=0xffbff514, input=0x637c8) at mail-storage.c:1652 box = (struct mailbox *) 0x594e8 ret = 0 #12 0xff23f138 in mail_storage_try_copy (_ctx=0xffbff514, mail=0x54cd8) at mail-copy.c:68 ctx = (struct mail_save_context *) 0x5e6e0 pmail = (struct mail_private *) 0x54cd8 input = (struct istream *) 0x637c8 from_envelope = 0x13d90 "MAILER-DAEMON" guid = 0xff2f0ec0 "" received_date = -1 #13 0xff23f23c in mail_storage_copy (ctx=0x5e6e0, mail=0x54cd8) at mail-copy.c:93 No locals. #14 0xff24ec28 in mailbox_copy (_ctx=0xffbff670, mail=0x54cd8) at mail-storage.c:1721 ctx = (struct mail_save_context *) 0x5e6e0 box = (struct mailbox *) 0x594e8 keywords = (struct mail_keywords *) 0x0 ret = 389032 #15 0xff24ec98 in mailbox_save_using_mail (ctx=0xffbff670, mail=0x54cd8) at mail-storage.c:1730 No locals. #16 0xff388070 in mail_deliver_save (ctx=0xffbff8a8, mailbox=0x13fe8 "INBOX", flags=0, keywords=0x0, storage_r=0xffbff83c) at mail-deliver.c:317 open_ctx = {user = 0x3d3a8, lda_mailbox_autocreate = true, lda_mailbox_autosubscribe = false} box = (struct mailbox *) 0x594e8 trans_flags = MAILBOX_TRANSACTION_FLAG_EXTERNAL t = (struct mailbox_transaction_context *) 0x5de88 save_ctx = (struct mail_save_context *) 0x0 headers_ctx = (struct mailbox_header_lookup_ctx *) 0x0 kw = (struct mail_keywords *) 0x0 error = MAIL_ERROR_NONE mailbox_name = 0x13fe8 "INBOX" errstr = 0x0 guid = 0xff3f73b0 "" changes = {pool = 0x13e38, uid_validity = 0, saved_uids = {arr = {buffer = 0x13e28, element_size = 1}, v = 0x13e28, v_modifiable = 0x13e28}, ignored_modseq_changes = 4282350008, changed = false} range = (const struct seq_range *) 0xff1d3580 default_save = true ret = 0 __FUNCTION__ = "mail_deliver_save" #17 0xff38869c in mail_deliver (ctx=0xffbff8a8, storage_r=0xffbff83c) at mail-deliver.c:403 ret = -1 #18 0x00012d08 in main (argc=3, argv=0xffbff964) at main.c:434 set_roots = {0x24b48, 0x0} ctx = {pool = 0x2eaf0, set = 0x30440, session = 0x2eb00, dup_ctx = 0x0, session_id = 0x0, src_mail = 0x54cd8, src_envelope_sender = 0x0, dest_user = 0x3d3a8, dest_addr = 0x25828 "testuser@domain", final_dest_addr = 0x25828 "testuser@domain", dest_mailbox_name = 0x13fe8 "INBOX", dest_mail = 0x5efa8, var_expand_table = 0x0, tried_default_save = true, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = 0xffbffad0 "testuser" errstr = 0xff3f48e8 "" path = 0x0 storage_service = (struct mail_storage_service_ctx *) 0x2f650 service_user = (struct mail_storage_service_user *) 0x2fe88 service_input = {module = 0x13fd0 "lda", service = 0x13fd0 "lda", username = 0xffbffad0 "testuser", session_id = 0x0, local_ip = {family = 0, u = {ip6 = { _S6_un = {_S6_u8 = '\0' <repeats 15 times>, _S6_u32 = {0, 0, 0, 0}, __S6_align = 0}}, ip4 = {S_un = {S_un_b = {s_b1 = 0 '\0', s_b2 = 0 '\0', s_b3 = 0 '\0', s_b4 = 0 '\0'}, S_un_w = {s_w1 = 0, s_w2 = 0}, S_addr = 0}}}}, remote_ip = {family = 0, u = {ip6 = { _S6_un = {_S6_u8 = '\0' <repeats 15 times>, _S6_u32 = {0, 0, 0, 0}, __S6_align = 0}}, ip4 = {S_un = {S_un_b = {s_b1 = 0 '\0', s_b2 = 0 '\0', s_b3 = 0 '\0', s_b4 = 0 '\0'}, S_un_w = {s_w1 = 0, s_w2 = 0}, S_addr = 0}}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x39330 user_source = 0x13f30 "" destaddr_source = 0x13f30 "" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE Joseph Tam <tam(a)math.ubc.ca>

2 2

[Dovecot] Too many imap connections in state "idling"
by 3.listas＠adminlinux.com.br 19 Dec '12

19 Dec '12

Hi, I have a "Ubuntu12.04 + dovecot-2.0.19" configuration in my server. My mailbox server is shared by ~ 10k domains. It works fine with ~50k accounts. On average, my server has about 6K simultaneous imap connections during office hours. But I've noticed that this server has an average of approximately 4k connections in state "idling". On other Dovecot servers the average connections in state "idling" is only 10%. My doveconf -n: http://thiagoh.prv.digirati.com.br/doveconf-n.txt Can someone help me? Thanks -- Thiago Henrique adminlinux.com.br

4 5

[Dovecot] Dsync help, new attempt
by Torbjorn Astlind 19 Dec '12

19 Dec '12

I observe that dsync fills Maildir correctly incl. cur, if I delete Maildir just before dsync runs. Can the failed mbox to Maildir conversion be caused by dovecot creating Maildir automatically if a client connects to dovecot? Should dovecot be shutdown when running dsync? All the best, Torbjörn 9. dsync help? (Torbjorn Astlind) ------------------------------ Message: 9 Date: Mon, 10 Dec 2012 16:33:08 +0100 From: Torbjorn Astlind <torbjorn(a)dbb.su.se> To: dovecot(a)dovecot.org Subject: [Dovecot] dsync help? Message-ID: <50C600B4.4040603(a)dbb.su.se> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi all, I have dovecot serving aprox 200 users through mbox, but the time has come to migrate to a new server, and to use Maildir. Using rsync from the old server I get all user data, i.e /home/*, and inboxes i.e /var/mail/* to a new server. This has worked well for some time as backup. The dsync should convert everything to Maildir, and it does... to an extent. (linebrakes for easy viewing) What I see, is that often, the /home/mail/xxx is NOT resulting in /home/xxx/Maildir/cur All mailboxes from /home/xxx, are properly converted to /home/xxx/Maildir/.name_of_ box Am I misunderstanding something? I get it to work sometimes if I delete the ~/Maildir completely before doing the dsync. All, and any help most welcome, Torbj?rn for i in `cat /home2/torbjorn/SCRIPTS/userlist.txt`; do dsync -D -v -f -u $i -c /etc/dovecot/alt_config/10-mail.MBOX.conf backup maildir:~/Maildir; done > /home2/torbjorn/SCRIPTS/mb2mdRES.txt 2>&1 the -c 10.mail.MBOX.conf contains, the parts I think are relevant for the dsync operation: mail_location = mbox:~/home:INBOX=/home/mail/%u namespace { type = private separator = '/' inbox = yes } dsync(root): Debug: Effective uid=501, gid=501, home=/home/torbjorn dsync(root): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/home:INBOX=/home/mail/torbjorn dsync(root): Debug: fs: root=/home/torbjorn/home, index=, control=, inbox=/home/mail/torbjorn, alt= dsync(root): Debug: Effective uid=501, gid=501, home=/home/torbjorn dsync(root): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir dsync(root): Debug: maildir++: root=/home/torbjorn/Maildir, index=, control=, inbox=/home/torbjorn/Maildir, alt= dsync(root): Debug: Namespace : /home/torbjorn/Maildir doesn't exist yet, using default permissions dsync(root): Debug: Namespace : Using permissions from /home/torbjorn/Maildir: mode=0700 gid=-1 dsync(torbjorn): Debug: Namespace : Using permissions from /home/torbjorn/home: mode=0700 gid=-1 ------------------------------ _______________________________________________ dovecot mailing list dovecot(a)dovecot.org http://dovecot.org/cgi-bin/mailman/listinfo/dovecot End of dovecot Digest, Vol 116, Issue 16 ****************************************

1 0

[Dovecot] Advice on sieve vacation for aliases
by Marcio Merlone 19 Dec '12

19 Dec '12

Greetings, I have a working mail server with sieve 1:1.2.9-1ubuntu6.5 working just fine. My commercial dept. has asked me to set a vacation message for comercial(a)domain.tld, but this is an alias to jhon(a)domain.tld and meggie(a)domain.tld. I have created a sieve_global_path = /etc/dovecot/global.sieve and it works, but not the intended way: it sends two vacation messages, each From: each employee, and not From: comercial(a)domain.tld alias. Is there how to make dovecot send only one vacation message From: comercial(a)domain.tld or will I have to survive with one message from jhon and another from meggie? Also, is there how to have a dovecot.lda-dupes for the alias instead the users? /etc/dovecot/global.sieve: require ["vacation"]; # rule:[Ferias coletivas] if anyof (header :contains "To" "comercial(a)domain.tld") { vacation :days 7 :subject "Férias coletivas de 20/12/2012 a 07/01/2013" text: bla bla bla. ; stop; } Thanks in advance and nice holidays. -- Marcio Merlone

2 1

[Dovecot] Adding a parameter to dovecot's deliver to get a meaningful return value
by Cyril RUSSO 18 Dec '12

18 Dec '12

Hi, First, please forgive me as I'm not subscribed to the list. I'm using Dovecot 2 (2.1.7) for serving my local maildir. This maildir is filled by an external process, fetching & merging mails from different sources that's outside this scope (but it's working). Dovecot is thus acting as a MTA, but more as a MDA. I've a web client running on the server, and it connects to Dovecot to get the mails. I've configured this web client & dovecot to use sieve to filter the mails. I know that since Dovecot does not actually deliver the messages, it's not the expected behavior. Anyway, I can use the "deliver" software in your suite for passing sieve's rules on each message in my maildir, and it's almost working. I've a cron job that run deliver on each message of my maildir, but I'm a bit puzzled now. Basically, I'm not able to figure out if a message was matched by a sieve filter or not. I'm a bit disappointed by two things: 1) Sieve's reject doesn't make "deliver" to return an error, so a sieve rule chain ending in "reject" can't be detected from my cron's script. I wonder it's due to the step deliver is supposed to be run (after a mail is already accepted). 2) "deliver" doesn't have a method to find out if it processed the given mail as input. Basically, would it be possible to either add a parameter to "deliver" (for example: "--fail-on-discarding") so it would fail if the sieve rules discard or reject (/ereject) the mail ? That way, I could detect the failure in my script, and know that I don't need to delete the initial mail since it was not filtered. Or, thought differently, I wonder if "deliver" could open a socket/pipe/file whatever, and write the number of processed mails in there (or using a "--return-processed-mail"), so a sieve rule ending with "delete", would be detectable by my script. Best regards, -- Cyril

2 1

Re: [Dovecot] Undeserved permissions error with g+s folder permissions
by tlhackque 18 Dec '12

18 Dec '12

> It's an MBox file that I'm trying to rename, not a directory. > | file Vendors/|| > ||Vendors/: setgid directory|| > || file Vendors/Test|| > ||Vendors/Test: empty|| > | > (Perhaps you missed that I said I created it "messages only", and the > config I originally posted showed the file storage is mbox...) > > So perhaps dovecot is confused - expecting an mbox file to have > directory permissions, when the directory it's in is setgid? At least > that's how it has seemed to me all along... > > Thanks. > > This communication may not represent my employer's views, > if any, on the matters discussed. > On 18-Dec-12 14:28, Reindl Harald wrote: >> Am 18.12.2012 19:39, schrieb Timo Sirainen: >>> On Sat, 2012-12-01 at 10:59 -0500, tlhackque wrote: >>>> It contains a subdirectory: >>>> >>>> -rw-rw---- 1 thlackque mail 84805345 Dec 1 10:34 Vendors/AVendor >>>> >>>> If I try to rename the AVendor folder (client is Thunderbird 17.0), I >>>> get this error: >>>> >>>> Unable to rename across conflicting directory permisssions >> this is 660 which is simply wrong for a directrory >> in case of diretcories you need the excue flag means 770 >> > -- This communication may not represent my employer's views, if any, on the matters discussed.

1 0

[Dovecot] Clear-text ldap passwords: {CLEAR} scheme?
by Kristof Bajnok 18 Dec '12

18 Dec '12

Hi, if I read the documentation correctly, the clear-text password storage scheme in Dovecot is called PLAIN. Unfortunately, in Sun LDAP Directory Server I have to specify them as "{CLEAR}" [1]. Is it somehow possible to make Dovecot recognise "{clear}foobar" passwords as plaintext? Now I get: auth: Error: ldap(username): Unknown scheme clear Thanks, Kristof [1]: http://docs.oracle.com/cd/E19850-01/816-6699-10/confattr.html#passwordStora…

2 1

[Dovecot] dovecot-lda (2.1.12) segfaults
by Joseph Tam 18 Dec '12

18 Dec '12

We uograded our dovecot from version 2.1.10 -> 2.10.12, but within the first hour of use, dovecot-lda would segfault during delivery. This left a lock file lying around causing the user's mail readers to hang; much hilarity ensues. All further deliveries to the same user will result in a crash. The only thing that fixes this condition is to blow away the user's INBOX cache index. Reindexing with "doveadm index" does not fix anything. I've installed a test version with one user's problematic INBOX cache indices and ran gdb dovecot-lda (gdb) run -t testuser </dev/null and it crashes with the following backtrace Program received signal SIGSEGV, Segmentation fault. 0xff2a0440 in mail_cache_header_fields_read (cache=0x5c120) at mail-cache-fields.c:323 323 if (field_hdr->size < sizeof(*field_hdr) + (gdb) bt #0 0xff2a0440 in mail_cache_header_fields_read (cache=0x5c120) at mail-cache-fields.c:323 #1 0xff29e8bc in mail_cache_compress_locked (cache=0x5c120, trans=0x63ce8, unlock=0xffbfeeef) at mail-cache-compress.c:361 #2 0xff29efd0 in mail_cache_compress (cache=0x5c120, trans=0x63ce8) at mail-cache-compress.c:489 #3 0xff2a3df4 in mail_cache_transaction_compress (ctx=0x5dac0) at mail-cache-transaction.c:180 #4 0xff2a4084 in mail_cache_transaction_open_if_needed (ctx=0x5dac0) at mail-cache-transaction.c:241 #5 0xff2a6e60 in mail_cache_field_want_add (ctx=0x5dac0, seq=1, field_idx=12) at mail-cache-transaction.c:1048 #6 0xff27e8e8 in index_mail_parse_header_register_all_wanted (mail=0x5e6b0) at index-mail-headers.c:175 #7 0xff27ec90 in index_mail_parse_header_init (mail=0x5e6b0, headers=0x0) at index-mail-headers.c:230 #8 0xff27f5c8 in index_mail_cache_parse_init (_mail=0x5e6b0, input=0x63760) at index-mail-headers.c:376 #9 0xff2299cc in mbox_save_get_input_stream (ctx=0x5dde8, input=0x62ed0) at mbox-save.c:411 #10 0xff22a084 in mbox_save_begin (_ctx=0x5dde8, input=0x62ed0) at mbox-save.c:520 #11 0xff24e9c0 in mailbox_save_begin (ctx=0xffbff514, input=0x62ed0) at mail-storage.c:1652 #12 0xff23f138 in mail_storage_try_copy (_ctx=0xffbff514, mail=0x54cd8) at mail-copy.c:68 #13 0xff23f23c in mail_storage_copy (ctx=0x5dde8, mail=0x54cd8) at mail-copy.c:93 #14 0xff24ec28 in mailbox_copy (_ctx=0xffbff670, mail=0x54cd8) at mail-storage.c:1721 #15 0xff24ec98 in mailbox_save_using_mail (ctx=0xffbff670, mail=0x54cd8) at mail-storage.c:1730 #16 0xff388070 in mail_deliver_save (ctx=0xffbff8a8, mailbox=0x13fe8 "INBOX", flags=0, keywords=0x0, storage_r=0xffbff83c) at mail-deliver.c:317 #17 0xff38869c in mail_deliver (ctx=0xffbff8a8, storage_r=0xffbff83c) at mail-deliver.c:403 #18 0x00012d08 in main (argc=3, argv=0xffbff964) at main.c:434 I dumped the value of field_hdr, and it has some bogus value (gdb) p field_hdr $1 = (const struct mail_cache_header_fields *) 0x20 Other observations - blowing away the indices does not fix it permanantly. The the LDA will eventually start crashing again. - the problematic indices do not cause problems with 2.1.10. Reverting to 2.1.10 restores proper mail delivery. Is this enough debug info to work your magic, Timo? I can supply the cache data if you neded it. It's compiled on a Solaris10. Joseph Tam <jtam.home(a)gmail.com>

2 1