Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
in working mailservers end to end encryption is/should be state of the art with smime/gpg etc
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
On 25 March 2013 12:30, Robert Schetterer rs@sys4.de wrote:
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers.
Simon
Am 25.03.2013 14:24, schrieb Simon Brereton:
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers
this is independent of the used software and transparent http://www.hermann-uwe.de/blog/howto-disk-encryption-with-dm-crypt-luks-and-...
Dear list,
we're using dovecot 2.1.15 (debian binary package). The following error can be found in the mail log files:
Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=<USER>, method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session=<In7mV8DYFQCNNQQ4> Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: read(/var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS) failed: Input/output error (FETCH for mailbox INBOX UID 25250) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Disconnected: Internal error occurred. Refer to server log for more information. [2013-03-25 15:08:46] in=246 out=953
The same problem was reported by Ralf Hildebrandt one year ago. The bug should be fixed with revision 3599790da3d7 but it seems to be there again.
Input/output errors of the file system are improbable because all files are accessible and can be read with cat and less.
Any ideas?
Best regards, Gordon
Leiter AG Technische Infrastruktur und Basisdienste Universitaetsrechenzentrum (URZ) E.-M.-Arndt-Universitaet Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany
Tel. +49 3834 86-1456 Fax. +49 3834 86-1401
Am 25.03.2013 15:22, schrieb Gordon Grubert:
Dear list,
we're using dovecot 2.1.15 (debian binary package). The following error can be found in the mail log files:
Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=<USER>, method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session=<In7mV8DYFQCNNQQ4> Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS
Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS
Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Corrupted index cache file /var/vmail/uni-greifswald.de/USER/Maildir/dovecot.index.cache: Broken physical size for mail UID 25250 Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: read(/var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS) failed: Input/output error (FETCH for mailbox INBOX UID 25250) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Disconnected: Internal error occurred. Refer to server log for more information. [2013-03-25 15:08:46] in=246 out=953
The same problem was reported by Ralf Hildebrandt one year ago. The bug should be fixed with revision 3599790da3d7 but it seems to be there again.
Input/output errors of the file system are improbable because all files are accessible and can be read with cat and less.
Any ideas?
Best regards, Gordon
please reread the list archive ,solutions where massive posted and a new repair script was created
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Am 25.03.2013 15:27, schrieb Robert Schetterer:
please reread the list archive ,solutions where massive posted and a new repair script was created
We did that before, for sure.
But there are NO working solutions for that and the problem still exists and ist a massive problem, because a simple version upgrade doesn't work and leads to a DOS of the infected systems.
The repair script hasn't worked at all with our kind of Maildir-Filenames.
If others run into the same problem:
We used this simple piece of code (which is much easier to read and adapt):
for FILE in * ; do
OLDNAME=$FILE
SIZE=zcat $FILE | wc -c
NEWNAME=echo $FILE | sed "s/\(.*\)S=.*:\(.*\)/\1S=$SIZE:\2/g"
if [ ! $OLDNAME = $NEWNAME ] ; then
echo mv "$OLDNAME" "$NEWNAME"
fidone
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Am 27.03.2013 17:51, schrieb Peer Heinlein:
Am 25.03.2013 15:27, schrieb Robert Schetterer:
please reread the list archive ,solutions where massive posted and a new repair script was created
We did that before, for sure.
But there are NO working solutions for that and the problem still exists and ist a massive problem, because a simple version upgrade doesn't work and leads to a DOS of the infected systems.
The repair script hasn't worked at all with our kind of Maildir-Filenames.
If others run into the same problem:
We used this simple piece of code (which is much easier to read and adapt):
for FILE in * ; do OLDNAME=$FILE SIZE=
zcat $FILE | wc -cNEWNAME=echo $FILE | sed "s/\(.*\)S=.*:\(.*\)/\1S=$SIZE:\2/g"if [ ! $OLDNAME = $NEWNAME ] ; then echo mv "$OLDNAME" "$NEWNAME" fidone
Peer
Hi Peer , as talked to Gordon, this was a total upgrade from 2.0.x to 2.1.x and you converted all maildir to compressed before, right ? posting some conf parameters might be helpfull , did you investigated broken maildirs for mixing compressed and uncompressed mails exist, as i understand Gordon there should be only compressed ? Have you checked about double compressed mails in broken maildirs ?
As wrote before, at my bug time after repair with script and upgrading dovecot, failures had gone and never returned, but my setup may be different from yours and failure did happend sometime upgrade dove 2.1.x not at migrate from dove 2.0.x, did you changed something other too ?
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
On 25.3.2013, at 16.22, Gordon Grubert gordon.grubert+lists@uni-greifswald.de wrote:
Mar 25 15:08:46 mailserver2 dovecot: imap-login: Login: user=<USER>, method=PLAIN, rip=IP, lip=IP, mpid=28663, TLS, session=<In7mV8DYFQCNNQQ4> Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Cached message size smaller than expected (2252 < 4821) Mar 25 15:08:46 mailserver2 dovecot: imap(USER): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS to /var/vmail/uni-greifswald.de/USER/Maildir/cur/1169635911.30322.scooby,S=2252:2,RS .. The same problem was reported by Ralf Hildebrandt one year ago. The bug should be fixed with revision 3599790da3d7 but it seems to be there again.
The Dovecot bug was fixed, but the real reason for this is that the S=values are wrong in your maildir. You can either run the fixing script or set maildir_broken_filename_sizes=yes.
Input/output errors of the file system are improbable because all files are accessible and can be read with cat and less.
That's just Dovecot's internal way of passing a failure to other parts of the code.
Am 25.03.2013 15:38, schrieb Timo Sirainen:
Hi,
The same problem was reported by Ralf Hildebrandt one year ago. The bug should be fixed with revision 3599790da3d7 but it seems to be there again.
The Dovecot bug was fixed, but the real reason for this is that the S=values are wrong in your maildir. You can either run the fixing script or set maildir_broken_filename_sizes=yes.
Looks like this (or a related) bug still exist.
If you have a Maildir-Storage with gzip compression enabled, everything's working fine if the user receives mail by LMTP. The mail is saved in his Maildir-Storage, having the right (uncompressed) size in the filename.
vmail vmail 1.9K Mar 26 22:17 1364332643.M527513P23361.mailserver2,S=3780,W=3860:2
But:
If the dovecot.index is broken, corrupt or deleted, Dovecot isn't able to rebuild his index-files.
In Step ONE dovecot creates his index-files, but looks like Dovecot's using the (smaller) FILEsize instead of the (larger) real size.
In Step TWO Dovecot's realizing that the cached size and the stored file size in the filename doesn't fit together. But Dovecot doesn't fix his index file; instead Dovecot's renaming the Maildir-Files, storing the (small) file size in the filename.
Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached message size smaller than expe cted (1467 < 3780)
Error: Maildir filename has wrong S value, r enamed the file from /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m ailserver2,S=3780,W=3860:2, to /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 13P23361.mailserver2,S=1856:2,
HOW TO REPRODUCE:
*) Create a Maildir-Store with zip enabled *) Deliver Mails into it. Everything's working fine, the filenames are right *) Delete dovecot.index* *) STEP ONE: Dovecot's complaining about broken index-files *) STEP TWO: Dovecot's renaming the files
I haven't seen any way to find to workaround or repair this broken Maildir-Storage. Even if I rename all files and set sizes in the filenames, Dovecot's complaining about the mismatch in his cache and starts his (broken) repair action.
If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Am 26.03.2013 23:15, schrieb Peer Heinlein:
If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files.
More and more users complained this morning about broken mailboxes and our logfile was full of errors.
We made a simple downgrade to 2.0.21 and now everything's working perfect.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On 27.3.2013, at 9.34, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Am 26.03.2013 23:15, schrieb Peer Heinlein:
If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files.
More and more users complained this morning about broken mailboxes and our logfile was full of errors.
We made a simple downgrade to 2.0.21 and now everything's working perfect.
maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug.
On 03/27/2013 08:38 AM, Timo Sirainen wrote:
On 27.3.2013, at 9.34, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Am 26.03.2013 23:15, schrieb Peer Heinlein:
If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files.
More and more users complained this morning about broken mailboxes and our logfile was full of errors.
We made a simple downgrade to 2.0.21 and now everything's working perfect.
maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug.
No, it does not.
Best regards, Gordon
Am 27.03.2013 11:19, schrieb Gordon Grubert:
On 03/27/2013 08:38 AM, Timo Sirainen wrote:
On 27.3.2013, at 9.34, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Am 26.03.2013 23:15, schrieb Peer Heinlein:
If we're right, this could be grow to a real problem. Every Server with zipped Maildirs can be completly ruined just by deleting his index-cache-files.
More and more users complained this morning about broken mailboxes and our logfile was full of errors.
We made a simple downgrade to 2.0.21 and now everything's working perfect.
maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug.
No, it does not.
Best regards, Gordon
agree, when i first run at that problem maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs manual by script, upgraded dovecot 2.1.x to newer version, that problem never came back again, just for interest what dovecot source did you use, did you you compile modifications by your own ?
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
On 03/27/2013 11:48 AM, Robert Schetterer wrote:
agree, when i first run at that problem maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs manual by script, upgraded dovecot 2.1.x to newer version, that problem never came back again, just for interest what dovecot source did you use, did you you compile modifications by your own ?
I used the debian binary package for dovecot 2.1.15 from xi.rename-it.nl
Best regards, Gordon
Am 27.03.2013 11:55, schrieb Gordon Grubert:
On 03/27/2013 11:48 AM, Robert Schetterer wrote:
agree, when i first run at that problem maildir_broken_file_sizes=yes didnt fixed it, i had to repair maildirs manual by script, upgraded dovecot 2.1.x to newer version, that problem never came back again, just for interest what dovecot source did you use, did you you compile modifications by your own ?
I used the debian binary package for dovecot 2.1.15 from xi.rename-it.nl
did you changed anything in your config too, using or changing other features too , while upgrade ? Did you modifications to the sources ( debian rules etc ) and recompile i.e integrate lucene etc ?
Best regards, Gordon
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Am 27.03.2013 11:19, schrieb Gordon Grubert:
We made a simple downgrade to 2.0.21 and now everything's working perfect.
maildir_broken_file_sizes=yes would also bring back v2.0 behavior. But yeah, looks like there's a bug.
No, it does not.
In Timo's first mail he wrote maildir_broken_fileNAME_sizes and we used that (which didn't helped at all).
mailserver2:~/dovecot.neu.2-1# grep -r maildir_broken * conf.d/10-mail.conf:maildir_broken_filename_sizes = yes mailserver2:~/dovecot.neu.2-1#
But maybe it's working with "maildir_broken_file_sizes" :-), we can test that on our test-system.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On 27.3.2013, at 0.15, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached message size smaller than expe cted (1467 < 3780)
Error: Maildir filename has wrong S value, r enamed the file from /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m ailserver2,S=3780,W=3860:2, to /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 13P23361.mailserver2,S=1856:2,
HOW TO REPRODUCE:
*) Create a Maildir-Store with zip enabled *) Deliver Mails into it. Everything's working fine, the filenames are right *) Delete dovecot.index* *) STEP ONE: Dovecot's complaining about broken index-files *) STEP TWO: Dovecot's renaming the files
Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size.
Am 27.03.2013 08:44, schrieb Timo Sirainen:
On 27.3.2013, at 0.15, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Mar 26 22:39:17 mailserver2 dovecot: imap(testuser): Error: Cached message size smaller than expe cted (1467 < 3780)
Error: Maildir filename has wrong S value, r enamed the file from /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M527513P23361.m ailserver2,S=3780,W=3860:2, to /var/vmail/uni-greifswald.de/testuser/Maildir/cur/1364332643.M5275 13P23361.mailserver2,S=1856:2,
HOW TO REPRODUCE:
*) Create a Maildir-Store with zip enabled
guess you mean zlib ?
*) Deliver Mails into it. Everything's working fine, the filenames are right *) Delete dovecot.index*
in fact i did this 2 weeks ago , no errors came up with 2.1.15, maildirs/mailboxes got work again
*) STEP ONE: Dovecot's complaining about broken index-files *) STEP TWO: Dovecot's renaming the files
Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size.
my problem was ,i couldnt find out why i needed to delete index* to get 2 Mailboxes work again,for more magic, no problem in the logs and mailboxes worked in thunderbird linux but not in thunderbird windows ( clean new setups ),i speculated to some problem with massive pop3 and imap in parallel from different ip at same time to the same mailbox via loadbalancers crashing something, but sadly couldnt reproduce it yet
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Am 27.03.2013 08:44, schrieb Timo Sirainen:
Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size.
Thats why I wrote from STEP ONE and from STEP TWO.
I had the impression, that Dovecot first rebuilds his cache with WRONG sizes and THEN starts in step two the renaming.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On 27.3.2013, at 18.41, Peer Heinlein p.heinlein@heinlein-support.de wrote:
Am 27.03.2013 08:44, schrieb Timo Sirainen:
Oh, except I actually forgot to load zlib plugin in my previous test. I can't reproduce with these steps.. and I don't really see why they would cause it anyway. A broken cached size would cause that rename, but not a missing cached size.
Thats why I wrote from STEP ONE and from STEP TWO.
I had the impression, that Dovecot first rebuilds his cache with WRONG sizes and THEN starts in step two the renaming.
Well, the question is then.. Why were the corrupted in the first place? Based on your previous error message it looked like the cache file contained the compressed size, so maybe zlib plugin wasn't loaded for some Dovecot process at that time?
Anyway, yeah, I guess there are two potential improvements here: Either a) don't rename maildir file if S=size is different from cached size or b) rename the S=size to the correct decompressed size (=no renaming if it's correct). Not sure which one is easier to do, possibly b) and possibly it should be done anyway.
In any case, I think this is a good addition: http://hg.dovecot.org/dovecot-2.2/rev/6d9444ea1c9a
Am 27.03.2013 20:10, schrieb Timo Sirainen:
Hi,
Well, the question is then.. Why were the corrupted in the first place? Based on your previous error message it looked like the cache file contained the compressed size, so maybe zlib plugin wasn't loaded for some Dovecot process at that time?
AFAIK zlib is always on:
mailserver2:~# doveconf | grep plugins mail_plugins = quota acl mail_log notify zlib mail_plugins = quota acl mail_log notify zlib sieve mail_plugins = quota acl mail_log notify zlib imap_quota imap_acl mail_plugins = quota acl mail_log notify zlib
I'll try to create and send you a test-case with an infected maildir.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
Hi,
in our case, the problem is solved since dovecot 2.2.13.
Best regards, Gordon
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 3/25/13 6:24 AM, Simon Brereton wrote:
On 25 March 2013 12:30, Robert Schetterer rs@sys4.de wrote:
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers.
Let's say you operate a mail server which uses a RAID array (or ZFS pool) as backend storage and one day one disks goes bad and needs to be replaced. You don't want information being leak from that bad disk when returning to vendor for replacement.
There are a lot of solutions to this issue. One possible way is to use FreeBSD's full disk encryption, geli(4), to encrypt all hard drives and have the email server hold the key on its boot partition, but don't protect it with a password so that the mail server can boot without any human intervention.
Encrypting individual user's mail store make little sense as one can still get your decryption key if they got root privilege, usually by tracing the login process or just replace it with something that can do the login but also save login credentials. In short, if root have been compromised, it's game over already.
Cheers,
-----BEGIN PGP SIGNATURE-----
iQEcBAEBCAAGBQJRUndLAAoJEG80Jeu8UPuzyyMIAJ22uv8U2OlZFFAUWTDL4zu/ tw6ZhxqQxhHVsg69kQPmIRVnMvlv0bhRqQphaJl5PQJAnfiwvrulx8ruFfTWIM3W xyxKMQtY/pJouRJwz1SZsfuuBNjU+ACX17IXIi5NDkLm8IT1FLgS9fWaYotACIUe 5fTXgodDDAGrWoYE4X1WTJiYCEE4UisilExaAJ0quk72NO/TzMnsLktR7mx0eSaP NqAi8ger9a2rflStgdJlI6pCmzRs4onAs2YWZq4F5Nv/wnnUysMsSjwNW+MuL4WY jWbX8oF+11kyH14vPLvzLKvMXjC9yKf8G880OPuMmgFQOrYAXzP5yp3w/rRVBCM= =SMvV -----END PGP SIGNATURE-----
If you are concerned about data being left on a hard drive when it fails and you are returning it to vendor, then I would consider hard drive degaussers. They are effective, but are very costly.
On Wed, Mar 27, 2013 at 12:36 AM, Xin Li delphij@delphij.net wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 3/25/13 6:24 AM, Simon Brereton wrote:
On 25 March 2013 12:30, Robert Schetterer rs@sys4.de wrote:
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers.
Let's say you operate a mail server which uses a RAID array (or ZFS pool) as backend storage and one day one disks goes bad and needs to be replaced. You don't want information being leak from that bad disk when returning to vendor for replacement.
There are a lot of solutions to this issue. One possible way is to use FreeBSD's full disk encryption, geli(4), to encrypt all hard drives and have the email server hold the key on its boot partition, but don't protect it with a password so that the mail server can boot without any human intervention.
Encrypting individual user's mail store make little sense as one can still get your decryption key if they got root privilege, usually by tracing the login process or just replace it with something that can do the login but also save login credentials. In short, if root have been compromised, it's game over already.
Cheers,
-----BEGIN PGP SIGNATURE-----
iQEcBAEBCAAGBQJRUndLAAoJEG80Jeu8UPuzyyMIAJ22uv8U2OlZFFAUWTDL4zu/ tw6ZhxqQxhHVsg69kQPmIRVnMvlv0bhRqQphaJl5PQJAnfiwvrulx8ruFfTWIM3W xyxKMQtY/pJouRJwz1SZsfuuBNjU+ACX17IXIi5NDkLm8IT1FLgS9fWaYotACIUe 5fTXgodDDAGrWoYE4X1WTJiYCEE4UisilExaAJ0quk72NO/TzMnsLktR7mx0eSaP NqAi8ger9a2rflStgdJlI6pCmzRs4onAs2YWZq4F5Nv/wnnUysMsSjwNW+MuL4WY jWbX8oF+11kyH14vPLvzLKvMXjC9yKf8G880OPuMmgFQOrYAXzP5yp3w/rRVBCM= =SMvV -----END PGP SIGNATURE-----
-- Daniel Reinhardt cryptodan@cryptodan.net http://www.cryptodan.net 301-875-7018(c) 410-455-0488(h)
Did anyone else get 13 identical copies of this response from Daniel???
On 2013-03-27 12:47 AM, Daniel Reinhardt cryptodan@gmail.com wrote:
If you are concerned about data being left on a hard drive when it fails and you are returning it to vendor, then I would consider hard drive degaussers. They are effective, but are very costly.
On Wed, Mar 27, 2013 at 12:36 AM, Xin Li delphij@delphij.net wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent
On 25 March 2013 12:30, Robert Schetterer rs@sys4.de wrote: the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers. Let's say you operate a mail server which uses a RAID array (or ZFS
On 3/25/13 6:24 AM, Simon Brereton wrote: pool) as backend storage and one day one disks goes bad and needs to be replaced. You don't want information being leak from that bad disk when returning to vendor for replacement.
There are a lot of solutions to this issue. One possible way is to use FreeBSD's full disk encryption, geli(4), to encrypt all hard drives and have the email server hold the key on its boot partition, but don't protect it with a password so that the mail server can boot without any human intervention.
Encrypting individual user's mail store make little sense as one can still get your decryption key if they got root privilege, usually by tracing the login process or just replace it with something that can do the login but also save login credentials. In short, if root have been compromised, it's game over already.
Cheers,
-----BEGIN PGP SIGNATURE-----
iQEcBAEBCAAGBQJRUndLAAoJEG80Jeu8UPuzyyMIAJ22uv8U2OlZFFAUWTDL4zu/ tw6ZhxqQxhHVsg69kQPmIRVnMvlv0bhRqQphaJl5PQJAnfiwvrulx8ruFfTWIM3W xyxKMQtY/pJouRJwz1SZsfuuBNjU+ACX17IXIi5NDkLm8IT1FLgS9fWaYotACIUe 5fTXgodDDAGrWoYE4X1WTJiYCEE4UisilExaAJ0quk72NO/TzMnsLktR7mx0eSaP NqAi8ger9a2rflStgdJlI6pCmzRs4onAs2YWZq4F5Nv/wnnUysMsSjwNW+MuL4WY jWbX8oF+11kyH14vPLvzLKvMXjC9yKf8G880OPuMmgFQOrYAXzP5yp3w/rRVBCM= =SMvV -----END PGP SIGNATURE-----
--
Best regards,
Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6224 | 678.514.6299 fax
nope
On Wed, 2013-03-27 at 07:23 -0400, Charles Marcus wrote:
Did anyone else get 13 identical copies of this response from Daniel???
On 2013-03-27 12:47 AM, Daniel Reinhardt cryptodan@gmail.com wrote:
If you are concerned about data being left on a hard drive when it fails and you are returning it to vendor, then I would consider hard drive degaussers. They are effective, but are very costly.
On 27 March 2013 05:36, Xin Li delphij@delphij.net wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 3/25/13 6:24 AM, Simon Brereton wrote:
On 25 March 2013 12:30, Robert Schetterer rs@sys4.de wrote:
Am 25.03.2013 11:03, schrieb Simon Brereton:
Hi
As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL.
What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore.
Is this possible? Is there a terrible reason to do it even if it is possible?
I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent.
Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely.
Thanks.
Simon
my meaning
crypted mailstore makes sense in a mail archive, in germany you have to have a mail archive for some kind of company emails all these solutions have some crypted mailstore , and some more features for data security, but thats a big theme, to big for here
crypt storage isnt "the saveness" per default, someone hacking the system and get root may hack your crypt storage too etc, also to big theme for here
Robert, indeed, this is sort of my point. If we encrypt laptop harddrives to prevent unauthorised access, that doesn't prevent the possiblity of someone who already has admin access to the device from decrypting/viewing/moving files. What it does do is prevent unauthorised access to the data if there is no admin access.
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers.
Let's say you operate a mail server which uses a RAID array (or ZFS pool) as backend storage and one day one disks goes bad and needs to be replaced. You don't want information being leak from that bad disk when returning to vendor for replacement.
There are a lot of solutions to this issue. One possible way is to use FreeBSD's full disk encryption, geli(4), to encrypt all hard drives and have the email server hold the key on its boot partition, but don't protect it with a password so that the mail server can boot without any human intervention.
Thanks. I think I will investigate this option. I use Debian, and I think the same approach is possible.
My concern with this approach is that if the drive is booted from then the information is freely available - but as you say, only if the root password is known. If the drive is simply mounted in different system, then the passphrase would be need (this is what I understand).
Alternatively, I could encrypt /var/mail/ and mount it as a LUKS volume to achieve the same effect. But I need a test plan and equipment.
Thanks for all the pointers.
Simon
[..]
Currently my mail store isn't encrypted and I would like to know if it is possible to do that, and if so, maybe get some pointers.
There are two main roads:
filesystem/disk based encryption
- Fast and easy to setup though (eg LUKS on Linux)
- does not protect against a running system being attacked, eg that they can run custom code in the same security level that thus can read the unencrypted content.
per-file encryption, eg with PGP/GnuPG
- Likely more complex to setup/fail-prone
- attacker getting access can only encrypt more mail and/or of course subvert any new mail, but can't decrypt old.
- there are a couple of tools which enable this, typically it is a procmail/pipe through gnupg
- Decryption of mails can be done with a "IMAP-proxy" style tool or possibly better/easier by the mail client.
- Check out:
For both:
- Store your decryption keys in a secure/offline place (cold-boot attacks)
- "Rubber Hose Crypto": http://www.schlockmercenary.com/2006-03-29
- "Lead Pipe Crypto": http://www.schlockmercenary.com/2009-10-19
Of course it always depends on the attack vectors that you are protecting against ;)
Greets, Jeroen
participants (11)
-
Charles Marcus
-
Daniel Reinhardt
-
Gordon Grubert
-
Jeroen Massar
-
Noel Butler
-
Peer Heinlein
-
Reindl Harald
-
Robert Schetterer
-
Simon Brereton
-
Timo Sirainen
-
Xin Li