How to use xapian with non-text attachments

[André Rodier]

Hello,

I am developing a hosting platform on Debian Bullseye, with Dovecot
amongst other tools.

I am trying to use the xapian full test search plugin, but I can see
the attachments are skipped:

This is what I have in the logs when running the indexing in verbose
mode:

-----------------------------------------------------------------------
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access
doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free =
66 MB
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query
doveadm(camille): Info: FTS Xapian: Query= uid:"44"
doveadm(camille): Info: FTS Xapian: Ngram(S) -> 63 items (total 0 KB)
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_unset_build_key
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Message-
Id,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_build_more
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access
doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free =
66 MB
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query
doveadm(camille): Info: FTS Xapian: Query= uid:"44"
doveadm(camille): Info: FTS Xapian: Ngram(XMID) -> 4 items (total 0 KB)
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_unset_build_key
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=X-
Mailer,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'xmailer'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=MIME-
Version,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'mimeversion'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-
Type,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contenttype'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Authentication-
Results,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'authenticationresults'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=X-AV-
Checked,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'xavchecked'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-
Type,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contenttype'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part
(Header=(null),Type=text/plain,Disposition=(null))
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_build_more
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_unset_build_key
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-
Type,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contenttype'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-
Description,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contentdescription'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-
Disposition,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contentdisposition'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part (Header=Content-Transfer-
Encoding,Type=(null),Disposition=(null))
doveadm(camille): Info: FTS Xapian: Unknown header (indexing)
'contenttransferencoding'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_build_key
doveadm(camille): Info: FTS Xapian: New part
(Header=(null),Type=text/csv,Disposition=attachment;
filename="file.csv")
doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv'
and disposition 'attachment; filename="file.csv"'
doveadm(camille): Info: FTS Xapian:
fts_backend_xapian_update_set_mailbox
doveadm(camille): Info: FTS Xapian: Unset box 'INBOX'
(c0d4e304584e5460dae30000075d7e67)
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_oldbox
doveadm(camille): Info: FTS Xapian: Done indexing 'INBOX'
(c0d4e304584e5460dae30000075d7e67) (13 msgs in 261 ms, rate: 49.8)
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release
(unset_box)
doveadm(camille): Info: FTS Xapian: Committed 'unset_box' in 17 ms
doveadm(camille): Info: FTS Xapian: Box is empty
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_deinit
(/home/users/camille/mails/indexes/xapian-indexes)
doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release
(update_deinit)
doveadm(camille): Info: FTS Xapian: Committed 'update_deinit' in 0 ms
doveadm(camille): Info: FTS Xapian: Deinit
/home/users/camille/mails/indexes/xapian-indexes)
-----------------------------------------------------------------------

Especially thos line:

doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv'
and disposition 'attachment; filename="file.csv"'

I have seen this as a bug on fts-xapian: 
https://github.com/grosjo/fts-xapian/issues/68

Which then references a PR on Dovecot core itself:

https://github.com/dovecot/core/pull/155

Any idea on how to use xapian with non-text attachments ?

Thanks a lot for your help!

André