I'm starting to think there is something I don't understand about running dovecot with SIS.

After I deleted the 3 emails that have the same attachment, the 3 attachments are still on disk (saved under the name of the hash).

After running doveadm purge & doveadm deduplicate, the 3 attachments are still on disk (the 3 emails were the only emails on server, so I can guarantee that no other emails are pointing to the attachments).

*What am I not understanding?* It makes no sense that the attachments would never be cleaned, so I am sure I am missing something.

Best regards,

Joe B

On Sat, Sep 5, 2015 at 11:40 AM, Joe Beaubien <joe.beaubien@gmail.com> wrote:

Hi everyone,

I am currently trying to setup our next dovecot server (2.2.18) with SIS enabled, but I have enconutered 1 problem and 1 question:

*1) Same attachment, different filenames*

I have a feeling a misconfigured something (or forgot to do something) because SIS almost works, but not quite since 3 identical attachment take 3x the space (see end of email for my SIS config).

For 3 the emails I sent with the same attachment, I got 3 seperate files with the same hash name but a different string following the dash:

-rw------- 1 info info 2.6M Sep 5 11:14 e35083e3280a21c6aaabbae8cb23d54493f514a5-6a22711a7b05eb5509480000045c9bf4 -rw------- 1 info info 2.6M Sep 5 11:16 e35083e3280a21c6aaabbae8cb23d54493f514a5-6c22711a7b05eb5509480000045c9bf4 -rw------- 2 info info 2.6M Sep 5 11:18 e35083e3280a21c6aaabbae8cb23d54493f514a5-6e22711a7b05eb5509480000045c9bf4 drwx------ 2 info info 4.0K Sep 5 11:18 hashes

For the record, the "hashes" subfolder has only 1 hash (e35083e3280a21c6aaabbae8cb23d54493f514a5) which perfectly matches the first part of the 3 filenames.

*2) Saving attachments under a generic name or more flexible permissions?*

It seems that dovecot saves the attachment with the username and group of the user who has received the email.

Is it possible to have dovecot save the attachments as a generic user (like dovecot or dovenull) so that all my different users can share the same pool of attachments?

Alternatively, is it possible to have dovecot save the attachment under more flexible permissions so that they can be shared between users?

This would allow us to save alot of disk space.

*Here is the SIS config I am using:*

mail_attachment_dir = /data/emails/attachments #mail_attachment_min_size = 128k #mail_attachment_fs = sis posix #mail_attachment_hash = %{sha1}

Any help would be very much appreciated.

Thank you,

Joe B

I figured out item #1, however, #2 is still problematic.

How can I setup dovecot to handle multiple mailboxes with SIS?

If I use "sis posix", the folders /var/attachments/[0-9a-f][0-9a-f] get created under the name of the first mailbox. Whenevver the second mailbox tries to read/write to those folders, it fails. Also, it seems that the attachments never get deduplicated, but i'll deal with that later.

If I use "sis-queue", then the files get deduplicated, but I cannot fix the multiple mailbox issues. I tried adding %u to the mail_attachment_dir, whcih seems to work at first. But when I try to deduplicate the attachments with "doveadm sis deduplicate" it seems that mail_attachment_fs does not support %u and the path does not resolve properly.

Any ideas?

Thanks,

Joe B

I'm trying to run SIS with queueing enabled

I tried running SIS with "sis posix", however when the /var/attachments/[0-9a-f][0-9a-f] folders get created under the name of the first mailbox, the second mailbox cannot write/read to them.

I then tried adding %u to mail_attachment_dir, but then I had errors when running doveadm sis deduplicate because mail_attachment_fs doesn't seem to support %u in the queue path.

How can I support multiple mailboxes with SIS???

On Sat, Sep 5, 2015 at 11:40 AM, Joe Beaubien <joe.beaubien@gmail.com> wrote:

Hi everyone,

I am currently trying to setup our next dovecot server (2.2.18) with SIS enabled, but I have enconutered 1 problem and 1 question:

*1) Same attachment, different filenames*

I have a feeling a misconfigured something (or forgot to do something) because SIS almost works, but not quite since 3 identical attachment take 3x the space (see end of email for my SIS config).

For 3 the emails I sent with the same attachment, I got 3 seperate files with the same hash name but a different string following the dash:

-rw------- 1 info info 2.6M Sep 5 11:14 e35083e3280a21c6aaabbae8cb23d54493f514a5-6a22711a7b05eb5509480000045c9bf4 -rw------- 1 info info 2.6M Sep 5 11:16 e35083e3280a21c6aaabbae8cb23d54493f514a5-6c22711a7b05eb5509480000045c9bf4 -rw------- 2 info info 2.6M Sep 5 11:18 e35083e3280a21c6aaabbae8cb23d54493f514a5-6e22711a7b05eb5509480000045c9bf4 drwx------ 2 info info 4.0K Sep 5 11:18 hashes

For the record, the "hashes" subfolder has only 1 hash (e35083e3280a21c6aaabbae8cb23d54493f514a5) which perfectly matches the first part of the 3 filenames.

*2) Saving attachments under a generic name or more flexible permissions?*

It seems that dovecot saves the attachment with the username and group of the user who has received the email.

Is it possible to have dovecot save the attachments as a generic user (like dovecot or dovenull) so that all my different users can share the same pool of attachments?

Alternatively, is it possible to have dovecot save the attachment under more flexible permissions so that they can be shared between users?

This would allow us to save alot of disk space.

*Here is the SIS config I am using:*

mail_attachment_dir = /data/emails/attachments #mail_attachment_min_size = 128k #mail_attachment_fs = sis posix #mail_attachment_hash = %{sha1}

Any help would be very much appreciated.

Thank you,

Joe B