I have this in my configuration:

protocol pop3 { pop3_enable_last = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls, oe-ns-eoh }

If I change the pop3_uidl_format I need to remove all the index file on my server ? I read on the doc that this change cann't be done on a production server ?

I have more than 10'000 accounts on this cluster so I can't have a big downtime.

Thanks

Le lundi 17 juillet 2006 à 14:48 +0300, Dumitru Negara a écrit :

Hi,

Dominique Feyer wrote:

...

I have a big problem with our cluster, if i read my mail with POP3 client (and check the option, keep a copy on server), every time I read my mail, my client retrive ALL messages an the mailbox !!!

I test this with:

• Apple Mail 1.0 + 2.0
• M$ Outlook 2000 + 2003
• Thunderbird

We use:

• Dovecot RC1
• Postfix on Debian Sarge (stable)

Can you know this bug ? Or have a workaround ?

This function is really important for our customer as we offer a backup solution for her mails.

Thanks

It's probably just a configuration issue.

If you upgraded from dovecot 0.99 or maybe migrated from another IMAP/POP3 server you should set correctly the "pop3_uidl_format" parameter.

I upgraded from 0.99 to 1.0.rc2 and below is my "pop3_uidl_format":

POP3 UIDL format to use. You can use following variables:

%v - Mailbox UIDVALIDITY

%u - Mail UID

%m - MD5 sum of the mailbox headers in hex (mbox only)

%f - filename (maildir only)

If you want UIDL compatibility with other POP3 servers, use:

UW's ipop3d : %08Xv%08Xu

Courier version 0 : %f

Courier version 1 : %u

Courier version 2 : %v-%u

Cyrus (<= 2.1.3) : %u

Cyrus (>= 2.1.4) : %v.%u

Older Dovecots : %v.%u

Note that Outlook 2003 seems to have problems with %v.%u format

which was

Dovecot's default, so if you're building a new server it would be a good

idea to change this. %08Xu%08Xv should be pretty fail-safe.

NOTE: Nowadays this is required to be set explicitly, since the old

default was bad but it couldn't be changed without breaking existing

installations. %08Xu%08Xv will be the new default, so use it for new

installations.

#pop3_uidl_format = pop3_uidl_format = %v.%u

This works fine even with Outlook 2000/2003. Clients (Thunderbird, OE, Outlook 2000/2003) download just new mails.

Regards, Dumitru

-- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

I have just use 0.99 for test before production on this server. The first version in production was beta9.

I just create some new account right now with RC2, and I have this probelm.

Thansk for your help

Le lundi 17 juillet 2006 à 15:02 +0300, Dumitru Negara a écrit :

Is this a new installation of Dovecot? Or you just upgraded from 0.99?

Dumitru

Dominique Feyer wrote:

...

I have this in my configuration:

protocol pop3 { pop3_enable_last = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls, oe-ns-eoh }

If I change the pop3_uidl_format I need to remove all the index file on my server ? I read on the doc that this change cann't be done on a production server ?

I have more than 10'000 accounts on this cluster so I can't have a big downtime.

Thanks

Le lundi 17 juillet 2006 à 14:48 +0300, Dumitru Negara a écrit :

...

Hi,

Dominique Feyer wrote:

...

I have a big problem with our cluster, if i read my mail with POP3 client (and check the option, keep a copy on server), every time I read my mail, my client retrive ALL messages an the mailbox !!!

I test this with:

• Apple Mail 1.0 + 2.0
• M$ Outlook 2000 + 2003
• Thunderbird

We use:

• Dovecot RC1
• Postfix on Debian Sarge (stable)

Can you know this bug ? Or have a workaround ?

This function is really important for our customer as we offer a backup solution for her mails.

Thanks

It's probably just a configuration issue.

If you upgraded from dovecot 0.99 or maybe migrated from another IMAP/POP3 server you should set correctly the "pop3_uidl_format" parameter.

I upgraded from 0.99 to 1.0.rc2 and below is my "pop3_uidl_format":

POP3 UIDL format to use. You can use following variables:

%v - Mailbox UIDVALIDITY

%u - Mail UID

%m - MD5 sum of the mailbox headers in hex (mbox only)

%f - filename (maildir only)

If you want UIDL compatibility with other POP3 servers, use:

UW's ipop3d : %08Xv%08Xu

Courier version 0 : %f

Courier version 1 : %u

Courier version 2 : %v-%u

Cyrus (<= 2.1.3) : %u

Cyrus (>= 2.1.4) : %v.%u

Older Dovecots : %v.%u

Note that Outlook 2003 seems to have problems with %v.%u format

which was

Dovecot's default, so if you're building a new server it would be a

good

idea to change this. %08Xu%08Xv should be pretty fail-safe.

NOTE: Nowadays this is required to be set explicitly, since the old

default was bad but it couldn't be changed without breaking existing

installations. %08Xu%08Xv will be the new default, so use it for new

installations.

#pop3_uidl_format = pop3_uidl_format = %v.%u

This works fine even with Outlook 2000/2003. Clients (Thunderbird, OE, Outlook 2000/2003) download just new mails.

Regards, Dumitru

-- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

I change nothing since 1 weeks, last update was to remove the ACL plugin.

For default_mail_env, I have:

default_mail_env = maildir:%h/:INDEX=MEMORY

For pop3_uidl_format, I read in the comment of the sample conf file:

"UW's ipop3d : %08Xv%08Xu" and a few line after "%08Xu%08Xv will be the new default" This is not the same string and I use the second "%08Xu% 08Xv"

This is strange the default Dovecot is %08Xu%08Xv and for UW is %08Xv% 08Xu ?

Thanks

Le lundi 17 juillet 2006 à 15:29 +0300, Dumitru Negara a écrit :

Did you modified something in your config recently? Like default_mail_env = ... CONTROL=/some/path

Dumitru

Dominique Feyer wrote:

...

I have just use 0.99 for test before production on this server. The first version in production was beta9.

I just create some new account right now with RC2, and I have this probelm.

Thansk for your help

Le lundi 17 juillet 2006 à 15:02 +0300, Dumitru Negara a écrit :

...

Is this a new installation of Dovecot? Or you just upgraded from 0.99?

Dumitru

Dominique Feyer wrote:

...

I have this in my configuration:

protocol pop3 { pop3_enable_last = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls, oe-ns-eoh }

If I change the pop3_uidl_format I need to remove all the index file on my server ? I read on the doc that this change cann't be done on a production server ?

I have more than 10'000 accounts on this cluster so I can't have a big downtime.

Thanks

Le lundi 17 juillet 2006 à 14:48 +0300, Dumitru Negara a écrit :

...

Hi,

Dominique Feyer wrote:

...

I have a big problem with our cluster, if i read my mail with POP3 client (and check the option, keep a copy on server), every time I read my mail, my client retrive ALL messages an the mailbox !!!

I test this with:

• Apple Mail 1.0 + 2.0
• M$ Outlook 2000 + 2003
• Thunderbird

We use:

• Dovecot RC1
• Postfix on Debian Sarge (stable)

Can you know this bug ? Or have a workaround ?

This function is really important for our customer as we offer a backup solution for her mails.

Thanks

It's probably just a configuration issue.

If you upgraded from dovecot 0.99 or maybe migrated from another IMAP/POP3 server you should set correctly the "pop3_uidl_format" parameter.

I upgraded from 0.99 to 1.0.rc2 and below is my "pop3_uidl_format":

POP3 UIDL format to use. You can use following variables:

%v - Mailbox UIDVALIDITY

%u - Mail UID

%m - MD5 sum of the mailbox headers in hex (mbox only)

%f - filename (maildir only)

If you want UIDL compatibility with other POP3 servers, use:

UW's ipop3d : %08Xv%08Xu

Courier version 0 : %f

Courier version 1 : %u

Courier version 2 : %v-%u

Cyrus (<= 2.1.3) : %u

Cyrus (>= 2.1.4) : %v.%u

Older Dovecots : %v.%u

Note that Outlook 2003 seems to have problems with %v.%u format

which was

Dovecot's default, so if you're building a new server it would be a

good

idea to change this. %08Xu%08Xv should be pretty fail-safe.

NOTE: Nowadays this is required to be set explicitly, since the old

default was bad but it couldn't be changed without breaking existing

installations. %08Xu%08Xv will be the new default, so use it for new

installations.

#pop3_uidl_format = pop3_uidl_format = %v.%u

This works fine even with Outlook 2000/2003. Clients (Thunderbird, OE, Outlook 2000/2003) download just new mails.

Regards, Dumitru

-- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

Le lundi 17 juillet 2006 à 15:57 +0300, Dumitru Negara a écrit :

1. You mentioned a "cluster". Can you provide more details on your setup?

We have 1 load balancer server, after 2 antispam antivirus, and finaly 2 dovecot server (for delivery of mail and POP/IMAP connection).

Share filesystem is on an NFS server and DB on an other PostgreSQL server.

2. Do you have "dovecot-uidlist" file in users homes? Is this file get updated after users download their emails over POP3?

Yes dovecot-uidlist is on the homedir of each user (NFS)

Before POP3 (with one message in the mailbox)

mail2:/var/mail/jamesdo.com/mails/test# stat dovecot-uidlist File: `dovecot-uidlist' Size: 106 Blocks: 8 IO Block: 4096 fichier r�gulier Device: eh/14d Inode: 45383789 Links: 1 Access: (0600/-rw-------) Uid: ( 2000/ UNKNOWN) Gid: ( 2000/ UNKNOWN) Access: 2006-07-17 15:09:15.000000000 +0200 Modify: 2006-07-17 13:30:01.000000000 +0200 Change: 2006-07-17 13:30:01.000000000 +0200

And after POP3

mail2:/var/mail/jamesdo.com/mails/test# stat dovecot-uidlist File: `dovecot-uidlist' Size: 106 Blocks: 8 IO Block: 4096 fichier r�gulier Device: eh/14d Inode: 45383789 Links: 1 Access: (0600/-rw-------) Uid: ( 2000/ UNKNOWN) Gid: ( 2000/ UNKNOWN) Access: 2006-07-17 15:12:54.000000000 +0200 Modify: 2006-07-17 13:30:01.000000000 +0200 Change: 2006-07-17 13:30:01.000000000 +0200

So the file is just accessed but not updated. If I send new mail, then this file is updated. The content of the file is:

1 0 5 1 1153135369.P21267Q0M227315.mail2.clm.net4all.ch 2 1153135573.P22034Q0M500053.mail2.clm.net4all.ch 3 1153141989.P14835Q0M535898.mail2.clm.net4all.ch 4 1153142047.P15076Q0M96504.mail2.clm.net4all.ch

Thanks

Dumitru

Dominique Feyer wrote:

...

I change nothing since 1 weeks, last update was to remove the ACL plugin.

For default_mail_env, I have:

default_mail_env = maildir:%h/:INDEX=MEMORY

For pop3_uidl_format, I read in the comment of the sample conf file:

"UW's ipop3d : %08Xv%08Xu" and a few line after "%08Xu%08Xv will be the new default" This is not the same string and I use the second "%08Xu% 08Xv"

This is strange the default Dovecot is %08Xu%08Xv and for UW is %08Xv% 08Xu ?

Thanks

Le lundi 17 juillet 2006 à 15:29 +0300, Dumitru Negara a écrit :

...

Did you modified something in your config recently? Like default_mail_env = ... CONTROL=/some/path

Dumitru

Dominique Feyer wrote:

...

I have just use 0.99 for test before production on this server. The first version in production was beta9.

I just create some new account right now with RC2, and I have this probelm.

Thansk for your help

Le lundi 17 juillet 2006 à 15:02 +0300, Dumitru Negara a écrit :

...

Is this a new installation of Dovecot? Or you just upgraded from 0.99?

Dumitru

Dominique Feyer wrote:

...

I have this in my configuration:

protocol pop3 { pop3_enable_last = no pop3_uidl_format = %08Xu%08Xv pop3_client_workarounds = outlook-no-nuls, oe-ns-eoh }

If I change the pop3_uidl_format I need to remove all the index file on my server ? I read on the doc that this change cann't be done on a production server ?

I have more than 10'000 accounts on this cluster so I can't have a big downtime.

Thanks

Le lundi 17 juillet 2006 à 14:48 +0300, Dumitru Negara a écrit : > Hi, > > Dominique Feyer wrote: >> I have a big problem with our cluster, if i read my mail with POP3 >> client (and check the option, keep a copy on server), every time I >> read >> my mail, my client retrive ALL messages an the mailbox !!! >> >> I test this with: >> >> - Apple Mail 1.0 + 2.0 >> - M$ Outlook 2000 + 2003 >> - Thunderbird >> >> We use: >> - Dovecot RC1 >> - Postfix on Debian Sarge (stable) >> >> Can you know this bug ? Or have a workaround ? >> >> This function is really important for our customer as we offer a >> backup >> solution for her mails. >> >> Thanks >> > It's probably just a configuration issue. > > If you upgraded from dovecot 0.99 or maybe migrated from another > IMAP/POP3 server you should set correctly the "pop3_uidl_format" > parameter. > > I upgraded from 0.99 to 1.0.rc2 and below is my "pop3_uidl_format": > > # POP3 UIDL format to use. You can use following variables: > # > # %v - Mailbox UIDVALIDITY > # %u - Mail UID > # %m - MD5 sum of the mailbox headers in hex (mbox only) > # %f - filename (maildir only) > # > # If you want UIDL compatibility with other POP3 servers, use: > # UW's ipop3d : %08Xv%08Xu > # Courier version 0 : %f > # Courier version 1 : %u > # Courier version 2 : %v-%u > # Cyrus (<= 2.1.3) : %u > # Cyrus (>= 2.1.4) : %v.%u > # Older Dovecots : %v.%u > # > # Note that Outlook 2003 seems to have problems with %v.%u format > which was > # Dovecot's default, so if you're building a new server it would be a > good > # idea to change this. %08Xu%08Xv should be pretty fail-safe. > # > # NOTE: Nowadays this is required to be set explicitly, since the old > # default was bad but it couldn't be changed without breaking > existing > # installations. %08Xu%08Xv will be the new default, so use it > for new > # installations. > # > #pop3_uidl_format = > pop3_uidl_format = %v.%u > > This works fine even with Outlook 2000/2003. Clients > (Thunderbird, OE, > Outlook 2000/2003) download just new mails. > > Regards, > Dumitru > >

-- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

On our servers, the line is added to the file dovecot-uidlist just when the server receive the mail (without POP3 download)

I have this before sending a mail:

data:/exports/mail/jamesdo.com/mails/test# cat dovecot-uidlist 1 0 2 1 1153144027.P23481Q0M210736.mail2.clm.net4all.ch

And after sending: data:/exports/mail/jamesdo.com/mails/test# cat dovecot-uidlist 1 0 3 1 1153144027.P23481Q0M210736.mail2.clm.net4all.ch 2 1153149315.P24388Q0M539227.mail1.clm.net4all.ch

If i check with find: data:/exports/mail/jamesdo.com/mails/test# find . . ./cur ./new ./new/1153149315.P24388Q0M539227.mail1.clm.net4all.ch ./tmp

The 3th line on my dovecot-uidlist is the new mail.

So are you sur that the file dovecot-uidlist is updated when clien check with POP ?

Thanks

Le lundi 17 juillet 2006 à 17:01 +0300, Dumitru Negara a écrit :

stat

Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

I did some testing.

On my servers I use maildrop as LDA. It does not update dovecot-uidlist on mail delivery (evidently, because it is a file created by dovecot).

I tested with dovecot deliver LDA and it indeed updates dovecot-uidlist on mail delivery.

Next step was to remove dovecot-uidlist and try to download mail with POP3. Result: Only new messages was downloaded. dovecot-uidlist was recreated by dovecot.

Then I deleted only dovecot indexes, without touching dovecot-uidlist. Result: Only new messages was downloaded. dovecot indexes were recreated by dovecot.

Then I deleted dovecot-uidlist and dovecot indexes. Result: All existing messages (new and seen) was downloaded. dovecot indexes and dovecot-uidlist were recreated by dovecot.

So, maybe somehow dovecot cannot find indexes and dovecot-uidlist file in user maildir and because of this all mails are downloaded every time. What logs are saying?

Regards, Dumitru

Dominique Feyer wrote:

On our servers, the line is added to the file dovecot-uidlist just when the server receive the mail (without POP3 download)

I have this before sending a mail:

data:/exports/mail/jamesdo.com/mails/test# cat dovecot-uidlist 1 0 2 1 1153144027.P23481Q0M210736.mail2.clm.net4all.ch

And after sending: data:/exports/mail/jamesdo.com/mails/test# cat dovecot-uidlist 1 0 3 1 1153144027.P23481Q0M210736.mail2.clm.net4all.ch 2 1153149315.P24388Q0M539227.mail1.clm.net4all.ch

If i check with find: data:/exports/mail/jamesdo.com/mails/test# find . . ./cur ./new ./new/1153149315.P24388Q0M539227.mail1.clm.net4all.ch ./tmp

The 3th line on my dovecot-uidlist is the new mail.

So are you sur that the file dovecot-uidlist is updated when clien check with POP ?

Thanks

Le lundi 17 juillet 2006 à 17:01 +0300, Dumitru Negara a écrit :

...

stat

For mmap_disable=yes my configuration is OK and I use lockd so not need lock_method=dotlock, I use lock_method=fcntl

For the cache, my configuration was not OK, so I update my fstab with:

10.10.0.1:/exports/mail /var/spool/mail nfs defaults,actimeo=0 0 0

I just add actimeo=0

But after this change I have always the same problem.

Now we have the index file created localy on the mail server (2 differents servers and indexes are not synch between this two server), can this be a problem ?

If i store the indexes on the NFS server, I'm affraid about the I/O usage !!!

Le mardi 18 juillet 2006 à 17:33 +1000, grant beattie a écrit :

On Mon, Jul 17, 2006 at 03:18:19PM +0200, Dominique Feyer wrote:

...

We have 1 load balancer server, after 2 antispam antivirus, and finaly 2 dovecot server (for delivery of mail and POP/IMAP connection).

Share filesystem is on an NFS server and DB on an other PostgreSQL server.

have you read http://wiki.dovecot.org/NFS ?

what OS/version is the NFS server? depending on your platform there may be settings you need to tweak.

grant.

-- Net4all S.A. Dominique Feyer Administrateur Système Ch. de la Colline 5bis CH-1007 Lausanne dfeyer@net4all.ch

Hello! Today, I tried to update Dovecot, initially everything pretended to be OK, but after update I see dovecot.log and found this line:

" dovecot: Jul 18 11:26:03 Error: IMAP(dario.a@telpin.com.ar): file istream-raw-mbox.c: line 485 (istream_raw_mbox_get_body_size): assertion failed: (rstream->mail_size != (uoff_t)-1) "

This line is repeated for many users, then I had to return to Dovecot-0.99.14-r1. This is my condiguration file for Dovecot-1.0.rc2:

" base_dir = /var/run/dovecot/ protocols = pop3 pop3s imap imaps listen = * disable_plaintext_auth = no shutdown_clients = yes log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.log ssl_listen = * ssl_disable = no ssl_cert_file = /etc/ssl/dovecot/dovecot.crt ssl_key_file = /etc/ssl/dovecot/dovecot.key login_chroot = yes default_mail_env = mbox:%h:INDEX=/etc/dovecot/dovecot-indexes%h:INBOX=%h.mbox mail_extra_groups = vmail mail_read_mmaped = yes mmap_disable = yes lock_method = dotlock valid_chroot_dirs = %h mail_save_crlf = no mbox_read_locks = dotlock mbox_write_locks = dotlock protocol imap { } protocol pop3 { pop3_uidl_format = %v%u } auth default { mechanisms = plain passdb sql { args = /etc/dovecot-mysql.conf } userdb sql { args = /etc/dovecot-mysql.conf } user = root } "

I am using NFS for Mboxes, no for INDEX Files. Please, could somebody help me? Thanks.

-- Dario Ale (dario.a en telpin.com.ar)