Dict proxy client returning empty string instead of multiline string

[Aki Tuomi]

> > > >>>>> On 10/16/2016 11:16 PM, Pierre Jaury wrote:
> > > >>>>>> Hello,
> > > >>>>>>
> > > >>>>>> I am using a dict proxy for my sieve extdata plugin to access some
> > > >>>>>> fields from an SQLite database (autoreply text and other
> > > >>>>>> database-configured items).
> > > >>>>>>
> > > >>>>>> All tests are performed against version 2.2.25.
> > > >>>>>>
> > > >>>>>>   $ dovecot --version
> > > >>>>>>   2.2.25 (7be1766)
> > > >>>>>>
> > > >>>>>> My configuration looks like:
> > > >>>>>>
> > > >>>>>>   dict {
> > > >>>>>>     sieve = sqlite:/etc/dovecot/pigeonhole-sieve.dict
> > > >>>>>>   }
> > > >>>>>>
> > > >>>>>>   [...]
> > > >>>>>>   sieve_extdata_dict_uri = proxy::sieve
> > > >>>>>>
> > > >>>>>> I am able to read pretty much any attribute without any issue, except
> > > >>>>>> when the value contains a special character like "\r" or "\n". By using
> > > >>>>>> the doveadm dict client, I narrowed it down to the dictionary management
> > > >>>>>> part (either server, protocol or client).
> > > >>>>>>
> > > >>>>>> I was suspecting escaping functions from "lib/strescape.c" (mostly
> > > >>>>>> str_tabescape and its counterpart, used by "lib-dict/client.c"), so I
> > > >>>>>> monitored socket communications. It seems that escaping is done properly
> > > >>>>>> on the server and the socket is not an issue either.
> > > >>>>>>
> > > >>>>>> The following strace dump results from running "doveadm dict get"
> > > >>>>>> against the dict socket:
> > > >>>>>>
> > > >>>>>>   connect(8, {sa_family=AF_UNIX, sun_path="..."}, 110) = 0
> > > >>>>>>   fstat(8, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
> > > >>>>>>   [...]
> > > >>>>>>   write(8, "H2\t0\t0\tadmin at domain.tld\tsieve\n", 30) = 30
> > > >>>>>>   [...]
> > > >>>>>>   read(8, "Otest\1r\1ntest\n", 8192)      = 14
> > > >>>>>>
> > > >>>>>> Indeed "\1r" and "\1n" are the escape sequences used by
> > > >>>>>> "lib/strescape.c". I went deeped and debugged the call to "dict_lookup"
> > > >>>>>> performed by doveadm. Indeed the client gets the proper string from the
> > > >>>>>> socket and to my surprise, it is properly unescaped.
> > > >>>>>>
> > > >>>>>> Then, in "client_dict_lookup" ("lib-dict/dict-client.c"), the call to
> > > >>>>>> "p_strdup" returns an empty string (null byte set at the target address).
> > > >>>>>>
> > > >>>>>> Before the call to the dict "->lookup" attribute (client_dict_lookup):
> > > >>>>>>
> > > >>>>>>    RAX: 0x7ffff73a37c0 (push   r14)
> > > >>>>>>    RBX: 0x6831b8 ("priv/reply_body")
> > > >>>>>>    RCX: 0x7fffffffe240 --> 0x682a60 --> 0x6831b8 ("priv/reply_body")
> > > >>>>>>    RDX: 0x6831b8 ("priv/reply_body")
> > > >>>>>>    RSI: 0x683288 --> 0x7ffff7653120 --> 0x7ffff73ea620 ([...])
> > > >>>>>>    RDI: 0x690ad0 --> 0x7ffff7400713 --> 0x75250079786f7270 ('proxy')
> > > >>>>>>
> > > >>>>>>    0x7ffff73a1f10 <dict_lookup+32>:	mov    rcx,r11 (value_r)
> > > >>>>>>    0x7ffff73a1f13 <dict_lookup+35>:	mov    rdx,r8 (key)
> > > >>>>>>    0x7ffff73a1f16 <dict_lookup+38>:	mov    rsi,r10 (pool)
> > > >>>>>>    0x7ffff73a1f19 <dict_lookup+41>:	mov    rdi,r9 (dict)
> > > >>>>>>    0x7ffff73a1f1c <dict_lookup+44>:	add    rsp,0x8
> > > >>>>>> => 0x7ffff73a1f20 <dict_lookup+48>:	jmp    rax
> > > >>>>>>
> > > >>>>>> Before the call to p_strdup in "client_dict_lookup":
> > > >>>>>>
> > > >>>>>>    RSI: 0x6832d8 ("test\r\ntest") (lookup.result.value)
> > > >>>>>>    RDI: 0x683288 --> 0x7ffff7653120 --> [...] (pool)
> > > >>>>>>    RAX: 0x0 (result)
> > > >>>>>>
> > > >>>>>>    0x7ffff73a384f:	nop
> > > >>>>>>    0x7ffff73a3850:	mov    rsi,QWORD PTR [rsp+0x8]
> > > >>>>>>    0x7ffff73a3855:	mov    rdi,r14
> > > >>>>>> => 0x7ffff73a3858:	call   0x7ffff736d3c0 <p_strdup at plt>
> > > >>>>>>    0x7ffff73a385d:	mov    QWORD PTR [r13+0x0],rax
> > > >>>>>>    0x7ffff73a3861:	mov    rsi,QWORD PTR [rsp+0x18]
> > > >>>>>>    0x7ffff73a3866:	xor    rsi,QWORD PTR fs:0x28
> > > >>>>>>    0x7ffff73a386f:	mov    eax,ebx
> > > >>>>>>
> > > >>>>>> After the call:
> > > >>>>>>
> > > >>>>>>    0x7ffff73a3850:	mov    rsi,QWORD PTR [rsp+0x8]
> > > >>>>>>    0x7ffff73a3855:	mov    rdi,r14
> > > >>>>>>    0x7ffff73a3858:	call   0x7ffff736d3c0 <p_strdup at plt>
> > > >>>>>> => 0x7ffff73a385d:	mov    QWORD PTR [r13+0x0],rax
> > > >>>>>>    0x7ffff73a3861:	mov    rsi,QWORD PTR [rsp+0x18]
> > > >>>>>>    0x7ffff73a3866:	xor    rsi,QWORD PTR fs:0x28
> > > >>>>>>    0x7ffff73a386f:	mov    eax,ebx
> > > >>>>>>    0x7ffff73a3871:	jne    0x7ffff73a38da
> > > >>>>>>
> > > >>>>>>    RSI: 0x0
> > > >>>>>>    RDI: 0x6832d8 --> 0x0
> > > >>>>>>    RAX: 0x6832d8 --> 0x0 (result)
> > > >>>>>>
> > > >>>>>> It is worth noting that I can reproduce the exact same execution flow
> > > >>>>>> with a non-multiline result string (lookup.result.value) that is
> > > >>>>>> properly copied by "p_strdup" and returned in RAX, then displayed by
> > > >>>>>> doveadm.
> > > >>>>>>
> > > >>>>>> I am not familiar with the pooling mechanism hidden behind the call to
> > > >>>>>> p_strdump and not quite sure why this behaviour is emerging. Maybe I am
> > > >>>>>> even miles away from an understanding of the issue here, but it sounds
> > > >>>>>> to me like something is wrong in the way "p_strdup" performs the copy.
> > > >>>>>>
> > > >>>>>> Hope this helps,
> > > >>>>>> kaiyou.
> > > >>>>>>
> > > >>>>>>
> > > >>>>> >

Fixed with https://github.com/dovecot/core/commit/4f051c3082080b9d69ef12c3720c683cff34b0da

Aki Tuomi