[Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[Timo Sirainen]

On 23.9.2013, at 16.52, megodin at inboxalias.com wrote:

>> I suspect your shell is breaking the 8bit chars into invalid UTF-8. I
>> just tried and it works fine with me.
> 
> Sorry, we got confusion here... apparently my (lousy) freemail provider 
> which I use to post on this mailinglist doesn't send some special 
> characters etc. unaltered exactly as I typed  - so I made a console 
> screenshot of the doveadm rename problem to make things clear.
> Please have a look here, it will reveal the doveadm rename problem 
> (and that my shell is working fine): 
> http://i.imgur.com/08wKji4.png

If doveadm says it's invalid UTF-8, it's invalid UTF-8. I guess your terminal isn't actually using UTF-8 then, but something else. ("locale" output should say something about UTF-8.) I guess doveadm could also automatically translate parameters to UTF-8, but that's a bit annoying to implement.

> 
>>> To get things worse, when renaming the mailfolder in Thunderbird to 
>>> "geschäftliches" it gets correctly written in the filesystem as 
>>> "gesch&AOQ-ftliches"- but afterwards it's NOT found anymore in the 
>>> dovecot index, e.g. when trying doveadm force-resync -u user it is 
>>> not listed anymore (untilback-renaming in fs to the name it was 
>>> indexed before).
> 
>> Folders aren't indexed by default (mailbox_list_index=no) and
>> force-resync doesn't affect them. Are you saying
>> "doveadm mailbox list -u user" doesn't show some folder? I'm not
>> aware of any bugs related to this.
> 
> No, the "doveadm mailbox list -u user" works just fine.
> 
> BUT - I'm still wondering about the resync issue - As I understand the 
> "doveadm force-resync -u user *" command rebuilds the index for the 
> MAILS within the mailfolders? If so, how can the resync be done if the 
> mailfolder itself is not found by doveadm anymore?

With mailbox_list_index=no there is no way a folder isn't found by doveadm.

>> I'd need to get some more specifics. Show show exists in filesystems
>> and what commands don't produce expected results (and what they 
>> produce instead).
> 
> For clearing up my point, I made another console screenshot with 
> red markings to point out the problem (bug). 
> Please have another look here:
> http://i.imgur.com/icPfys9.png

The problem here is that * is expanded by your shell, not doveadm. And it expands into Tr&-AOQ-sh as it's in the filesystem, but that's only the mUTF-7 encoding of it. The UTF-8 version of the name is Tr&AOQ-sh. So doveadm only sees that you attempted to resync a nonexistent mailbox. Using '*' with quotes would work, since doveadm would do the expansion then.