Hello again,
I am still stuck I'm afraid.
I now have doveadm backup working perfectly from a small gmail mailbox (a few hundred messages), but when I try the same configuration (apart from usernames and passwords obviously) with a large gmail mailbox (around 60,000 messages), doveadm connects successfully, replicates the gmail folder tree, then produces no further output until about 30 minutes later when the server times out and doveadm crashes. No actual messages are downloaded.
Does doveadm backup try by default to download all messages in a single transaction? And if so, is there a way to split it up?
Command output and configuration are at the end of this message. Dovecot version is 2.3.10.
Thanks in advance for any help.
Ben
ben@expectation:~$ doveadm backup -D -R -u ben imapc: dsync(ben): Info: imapc(imap.gmail.com:993): Connected to 74.125.133.109:993 (local MY_IP_ADDRESS:53244) dsync(ben): Warning: imapc(imap.gmail.com:993): Server disconnected with message: System Error - reconnecting (delay 0 ms) dsync(ben): Info: imapc(imap.gmail.com:993): Connected to 74.125.133.108:993 (local MY_IP_ADDRESS:34076) dsync(ben): Panic: file imapc-client.c: line 173 (imapc_client_run_pre): assertion failed: (client->ioloop == NULL) dsync(ben): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f94271610f2] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f942716120e] -> /usr/lib/dovecot/libdovecot.so.0(+0xe6aee) [0x7f942716baee] -> /usr/lib/dovecot/libdovecot.so.0(+0xe6b31) [0x7f942716bb31] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f94270c61bb] -> /usr/lib/dovecot/libdovecot-storage.so.0(imapc_client_run+0x158) [0x7f94274c27e8] -> /usr/lib/dovecot/libdovecot- storage.so.0(imapc_simple_run+0x3c) [0x7f94274bf8ac] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x94a23) [0x7f94274b7a23] -> /usr/lib/dovecot/libdovecot- storage.so.0(imapc_list_get_mailbox_flags+0x68) [0x7f94274b8288] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x98e69) [0x7f94274bbe69] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x98fed) [0x7f94274bbfed] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9bcc9) [0x7f94274becc9] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa27cb) [0x7f94274c57cb] -> /usr/lib/dovecot/libdovecot- storage.so.0(imapc_connection_input_pending+0x227) [0x7f94274c7dc7] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa4e1a) [0x7f94274c7e1a] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f9427184999] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x134) [0x7f9427186274] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f9427184a9c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f9427184cc8] -> /usr/lib/dovecot/libdovecot- storage.so.0(imapc_client_run+0xbb) [0x7f94274c274b] -> /usr/lib/dovecot/libdovecot- storage.so.0(imapc_mailbox_run_nofetch+0x20) [0x7f94274bf7a0] -> /usr/lib/dovecot/libdovecot-storage.so.0(imapc_mail_fetch+0x199) [0x7f94274b9419] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9772f) [0x7f94274ba72f] -> /usr/lib/dovecot/libdovecot- storage.so.0(mail_get_stream_because+0x64) [0x7f9427461fa4] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_mail_update_access_parts_post+0x7a) [0x7f94274e6c2a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xcaf3f) [0x7f94274edf3f] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_storage_search_next_nonblock+0x10d) [0x7f94274ee1cd] -> /usr/lib/dovecot/libdovecot- storage.so.0(mailbox_search_next_nonblock+0x22) [0x7f94274706e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next+0x3d) [0x7f942747074d] -> /usr/bin/doveadm(dsync_mailbox_export_next_mail+0x57) [0x562a19bf0597] Aborted (core dumped)
# 2.3.10 (0da0eff44): /etc/dovecot/dovecot.conf # OS: Linux 4.15.0-91-generic x86_64 Ubuntu 18.04.4 LTS # Hostname: expectation imapc_features = rfc822.size gmail-migration fetch-headers imapc_host = imap.gmail.com imapc_password = # hidden, use -P to show it imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no imapc_user = MY_GMAIL_ADDRESS mail_location = maildir:~/Maildir mail_prefetch_count = 20 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } protocols = " imap" service stats { unix_listener stats-reader { group = mail mode = 0660 user = dovecot } unix_listener stats-writer { group = mail mode = 0660 user = dovecot } } ssl = no ssl_client_ca_dir = /etc/ssl/certs userdb { driver = passwd }