"failed to map segment from shared object" in sieve pipe
Hello,
I have encountered a problem attempting to run a program in a sieve pipe. The program is a Perl script that loads a number of binary (XS) modules at runtime. The program crashes, it appears, if it tries to load too many. The (non-Perl part of the) error message is "failed to map segment from shared object” — no additional information.
Cryptic error messages like this have the odour of imposed resource restrictions, so I’m wondering if there are any on the pipe/filter/execute implementation (e.g. via program_client_run).
Any configuration knobs I can twiddle? Any insights for debugging?
PS this is the dovecot 2.2.22 (fe789d2), the one that ships with Ubuntu Xenial.
Regards,
-- Dorian Taylor Make things. Make sense. https://doriantaylor.com
Op 3/22/2018 om 12:14 AM schreef Dorian Taylor:
Hello,
I have encountered a problem attempting to run a program in a sieve pipe. The program is a Perl script that loads a number of binary (XS) modules at runtime. The program crashes, it appears, if it tries to load too many. The (non-Perl part of the) error message is "failed to map segment from shared object” — no additional information.
Cryptic error messages like this have the odour of imposed resource restrictions, so I’m wondering if there are any on the pipe/filter/execute implementation (e.g. via program_client_run).
Any configuration knobs I can twiddle? Any insights for debugging?
PS this is the dovecot 2.2.22 (fe789d2), the one that ships with Ubuntu Xenial.
Those knobs depend a on your config. If there is truly some resource limit causing this, it could be the vsz_limit of the (parent) process invoking your program. Which service is doing that depends on your config, either LMTP, the script service (when you're using socket programs), or dovecot-lda (so, basically your MTA - not controlled by Dovecot obviously).
https://wiki.dovecot.org/Services#Service_limits
Regards,
Stephan.
On Mar 21, 2018, at 5:16 PM, Stephan Bosch <stephan@rename-it.nl> wrote:
Those knobs depend a on your config. If there is truly some resource limit causing this, it could be the vsz_limit of the (parent) process invoking your program. Which service is doing that depends on your config, either LMTP, the script service (when you're using socket programs), or dovecot-lda (so, basically your MTA - not controlled by Dovecot obviously).
https://wiki.dovecot.org/Services#Service_limits
Regards,
Stephan.
Indeed it was vsz_limit. I set it to zero (under service lmtp) and now the script works.
However I’m noticing a couple things:
I previously had the vsz_limit unset, and subsequently set to 256M; neither worked.
I wrote a test script that just ran in a loop appending to a string and printing to stderr how long the string was; it was killed after it reported accumulating about 2MB.
Weird? or normal?
-- Dorian Taylor Make things. Make sense. https://doriantaylor.com
On 3/21/18 5:28 PM, Dorian Taylor wrote:
I previously had the vsz_limit unset, and subsequently set to 256M; neither worked.
I wrote a test script that just ran in a loop appending to a string and printing to stderr how long the string was; it was killed after it reported accumulating about 2MB.
Weird? or normal?
Probably normal. VSZ includes the total size of all shared objects mapped into the process address space, whether it's actually used or not; it's much larger than the memory actively used (which is somewhat related to "RSS").
We had to increase the Dovecot vsz_limit to 1 GB (!!!) because it would occasionally run out of 256 MB VSZ memory when doing FTS indexing at delivery time. A bunch of large Perl modules could conceivably do that, too.
I'd set it to 1 GB instead of zero, though, and see if that works. You probably want *SOME* limit in case of real problems.
-- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
On Mar 21, 2018, at 5:47 PM, Robert L Mathews <lists@tigertech.com> wrote:
Probably normal. VSZ includes the total size of all shared objects mapped into the process address space, whether it's actually used or not; it's much larger than the memory actively used (which is somewhat related to "RSS”).
Ahh yes of course. That makes sense.
We had to increase the Dovecot vsz_limit to 1 GB (!!!) because it would occasionally run out of 256 MB VSZ memory when doing FTS indexing at delivery time. A bunch of large Perl modules could conceivably do that, too.
This is just an address on a lonely internal box with a little attachment-ingesting bot-like thing that gets a message about once a week, so luckily not much needed in the concurrency department.
I'd set it to 1 GB instead of zero, though, and see if that works. You probably want *SOME* limit in case of real problems.
Noted. Good idea.
-- Dorian Taylor Make things. Make sense. https://doriantaylor.com
participants (3)
-
Dorian Taylor
-
Robert L Mathews
-
Stephan Bosch