slow mailbox refreshes

[Maciej Milaszewski IQ PL]

Hello
Fabian for tunning maby this help you:

- change type sort of roundcube

Problem: https://github.com/roundcube/roundcubemail/issues/3556
solved: https://github.com/roundcube/roundcubemail/issues/5072#ticket

- if you have many big account user solr for full-text search

Probably is problem with dovecot and dovecot give you timeout
- first get - you get timeout
- second all is ok



This is simple script to test it:

---------------------- test.py -------------------

#!/usr/bin/env python3

import imaplib
import time
import logging
import re

class IMAPLoginAdapter(logging.LoggerAdapter):

    def process(self, msg, kwargs):
        return '[%s] %s' % (self.extra['imap_login'], msg), kwargs

def timeit(method):

    def timed(*args, **kw):
        ts = time.time()
        result = method(*args, **kw)
        te = time.time()
        imap_logger.info("%s %2.2f sec", method.__name__, te - ts)
        return result

    return timed

@timeit
def test_search_all(imap):
    type_, data = imap.uid("search", None, "ALL")
    uids = data[0].split()
    imap_logger.info("%s %d", type_, len(uids))
    return uids

@timeit
def test_peek_index(imap, uids):
    page_uids = uids[:50]
    page_uids = b",".join(page_uids)
    type_, data = imap.uid("fetch", page_uids, "(INTERNALDATE
BODY.PEEK[HEADER.FIELDS (DATE)])")
    imap_logger.info("%s %d", type_, len(data))

@timeit
def test_size(imap):
    type_, data = imap.fetch("1:*", "(RFC822.SIZE)")
    size = 0
    for d in data:
        match = re.search(br"^[0-9]+ \(RFC822\.SIZE ([0-9]+)\)$", d)
        if not match:
            raise ValueError
        size += int(match.group(1))
    imap_logger.info("%s %d / %d", type_, size, len(data))

def imap_init(config):
    if config.has_option("IMAP", "port"):
        imap = imaplib.IMAP4_SSL(config.get("IMAP", "server"),
config.get("IMAP", "port"))
    else:
        imap = imaplib.IMAP4_SSL(config.get("IMAP", "server"))
    imap.login(config.get("IMAP", "login"), config.get("IMAP", "password"))
    imap.select("INBOX")
    return imap

def imap_close(imap):
    imap.close()
    imap.logout()

def main():
    import argparse
    import configparser
    parser = argparse.ArgumentParser()
    parser.add_argument("config_file", help="IMAP access config file")
    args = parser.parse_args()
    config = configparser.ConfigParser()
    config.read(args.config_file)
    imap_logger.extra['imap_login'] = config.get("IMAP", "login")
    imap = imap_init(config)
    # uids = test_search_all(imap)
    # test_peek_index(imap, uids)
    test_size(imap)
    imap_close(imap)

logging.basicConfig(
    # filename="test.log",
    level=logging.INFO,
    format="%(asctime)s %(message)s"
)

imap_logger = IMAPLoginAdapter(logging.getLogger(), {'imap_login': None})

if __name__ == "__main__":
    main()
-------------- end test.py -----------------

example.ini:

------------------ start example.ini ----------------

[IMAP]
server = imap.youserver.org
login = your_username
password = your_password

----------- stop example.ini -----------------


example:

./test.py example.ini
2018-07-09 16:58:48,324 [duzomailii at dasit1.foomydomain.org] OK 239440164
/ 20000
2018-07-09 16:58:48,325 [duzomailii at dasit1.foomydomain.org] test_size
0.29 sec




W dniu 09.07.2018 o 16:37, Fabian A. Santiago pisze:
> Hello,
>
> I am using dovecot 2.3.2 on my private email server in conjunction with:
>
> centos 7.5
> apache 2.4.6
> mariadb 10.2.16
> roundcube mail 1.3.6
> php 5.6.36
> postfix 2.10.1
>
>
> I have one mailbox with nearly 30k messages in it dispersed across
> several folders. it's often very slow in refreshing the message list,
> especially in the one largest 25k+ message folder. is this simply to
> be expected based on my message count or is there some kind of
> performance optimization and tuning i can do to improve the response
> times?
>
> The mail server itself is a linode hosted VPS:
>
> intel xeon e5-2680 v2 @ 2.80 GHz, 4 cores
> 8 GB real memory
> 2 GB virtual memory
> 200 GB local storage (ext4)
> KVM hypervisor
>
> Thanks everyone for any guidance you can offer.
>
>


-- 
Maciej Miłaszewski
IQ PL Sp. z o.o.
Administrator Systemowy