lmtp memory usage problem - Fatal: pool_system_realloc(268435456): Out of memory

[Arkadiusz Miśkiewicz]

Hi.

In my setup exim delivers mails to dovecot using LMTP. In one LMTP session exim
can deliver up to 200 recipients (batch_max set to that value).

Now the problem is that sometimes 256MB is not enoug for dovecot lmtp to handle
incoming emails.

My questions:
- how big memory limit should be for lmtp? I was thinking that lmtp (more or less) simply
reads from one descriptor and writes to file, then does rename() (maildir used here)
and that's all. That shouldn't require big number of memory. So how to determine correct
memory limit and what affects this limit?

- is number of recipients in one LMTP session important here? Not sure, maybe dovecot
stores email in memory first and then writes to the user maildirs? Setting (batch) limit to 1
could reduce memory usage then (since no need to store anything in memory) ?

Thanks,

Log:

Sep  4 16:10:30 mail dovecot: lmtp(21383, user): Fatal: pool_system_realloc(268435456): Out of memory
Sep  4 16:10:30 mail dovecot: lmtp(21383, user): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x682a0) [0x7fe327e632a0] -> /usr/lib64/dovecot/libd
ovecot.so.0(+0x6837e) [0x7fe327e6337e] -> /usr/lib64/dovecot/libdovecot.so.0(i_error+0) [0x7fe327e1dbf8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x7d6a3) [0x7fe327e786a3] ->
 /usr/lib64/dovecot/libdovecot.so.0(i_stream_grow_buffer+0x8f) [0x7fe327e6c4cf] -> /usr/lib64/dovecot/libdovecot.so.0(i_stream_try_alloc+0x82) [0x7fe327e6c592] -> /usr/lib6
4/dovecot/libdovecot.so.0(+0x73b9b) [0x7fe327e6eb9b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x73c36) [0x7fe327e6ec36] -> /usr/lib64/dovecot/libdovecot.so.0(i_stream_read+0x
53) [0x7fe327e6bad3] -> /usr/lib64/dovecot/libdovecot.so.0(+0x77391) [0x7fe327e72391] -> /usr/lib64/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7fe327e6bad3] -> /usr/lib
64/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7fe327e6c2fd] -> /usr/lib64/dovecot/libdovecot.so.0(io_stream_copy+0x7f) [0x7fe327e7cacf] -> /usr/lib64/dovecot/libdo
vecot.so.0(+0x83310) [0x7fe327e7e310] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_send_istream+0x4d) [0x7fe327e7c92d] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maild
ir_save_continue+0x5a) [0x7fe32811b14a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x88) [0x7fe328145328] -> /usr/lib64/dovecot/libdovecot-storage.so.0
(maildir_copy+0x42) [0x7fe3281179c2] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(+0xb52b) [0x7fe32764352b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_cop
y+0x6d) [0x7fe32814d29d] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x185) [0x7fe3283f1765] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0xeb) [0
x7fe3283f1b6b] -> dovecot/lmtp() [0x405d80] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x3f) [0x7fe327e7398f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handl
er_run_internal+0xd7) [0x7fe327e74897] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fe327e739f9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x3
8) [0x7fe327e73a78] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fe327e22ca3]
Sep  4 16:10:30 mail dovecot: lmtp(21383, user): Fatal: master: service(lmtp): child 21383 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB },             
 you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)

-- 
Arkadiusz Miśkiewicz, arekm / maven.pl