Hi Stan,

On 2010-06-26 03:02 UTC Stan Hoeppner wrote:

Steffen Kaiser put forth on 6/25/2010 7:01 AM:

...

2. thunderbird opened too many simultaneous connections to the server. I do not remember where they would blocked or terminated, but in some cases thunderbird did not seem to detect this failure

This is a pet peeve of mine. Recent TB revs default to opening 5 IMAP connections. Some time ago I did more than cursory testing and found that TBird only uses 1 of the 5. The other 4 just sit there idle, adding clutter to the process list and eating a small amount of RAM. If it weren't for Linux' share memory management these processes would be eating a whole ton of RAM on busy Dovecot servers.

The connections are used for IMAP IDLE [1], AFAIK. So the first five folders (a.k.a mailboxes) you access(?) get "push mail" - the moment a new mail goes in or out, Thunderbird knows about it. Why they chose the number five, I don't know. IMO it would be better, if you could choose explicitly, which folders should be "push", and then TB would create as many connections as you have set folders to "push". Or even better: IMAP NOTIFY [2] gets implemented in dovecot and in Thunderbird, and one TCP connection suffices for an arbitrary amount of "push" folders :)

Patrick.

[1] http://www.faqs.org/rfcs/rfc2177.html [2] http://www.faqs.org/rfcs/rfc5465.html

-- Key ID: 0x86E346D4 http://patrick-nagel.net/key.asc Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Sat, Jun 26, 2010 at 09:11:19PM -0500, Stan Hoeppner wrote:

Patrick Nagel put forth on 6/26/2010 2:08 AM:

[...]

I'm no IMAP expert, but what you state here doesn't make any sense at all.

[...]

postfix smtpd fire the email showed in my inbox in TB. This shows that IDLE is working with only a single IMAP connection to Dovecot. [...]

...

[1] http://www.faqs.org/rfcs/rfc2177.html [2] http://www.faqs.org/rfcs/rfc5465.html

RFCs are a huge PITA to read and digest. I may take a look if required, but for now I think this theory is malarky. No offense intended. Just calling it as I see it.

The references are spot-on. The IDLE command is just designed to notify changes to the *selected* mailbox. And a client can have just one selected mailbox (per-connection, that is). That's simply a limitation of the protocol. Clients may work around this by opening several connections and selecting one mailbox per connection.

And refusing to read 130 lines of RFC (the first one, describing IDLE is really that short) to just say "meh, I don't believe you" doesn't sound really appropriate.

Just my 2 cents.

Regards

• -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFMJtkfBcgs9XrR2kYRAkJrAJ9Qi4jkKkTqRZ2qdXfVUkdCf/gfiACeN0fn wQuhHOQlv26LaUh6pfc9dIU= =n5om -----END PGP SIGNATURE-----

On 06/27/2010 06:04 AM, Stan Hoeppner wrote:

Regardless, my point is valid and stands: there is no (good) reason for the protocol to require multiple socket connections when everything can be accomplished more efficiently (in terms of resources consumed) over a single socket. I'm sure many people more qualified than me have pointed this out WRT the IMAP protocol over the years.

Tomas is right. It's only possible to monitor one folder via IDLE per IMAP connection. It's stupid and inefficient, but that's how IMAP IDLE was designed.

Fortunately, there's the NOTIFY extension to overcome that limitation. But it's not supported in all clients (nor in all servers, I'd guess).

-- A visit to a strange place will bring fresh work.

Eduardo M KALINOWSKI eduardo@kalinowski.com.br

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Sun, Jun 27, 2010 at 04:04:39AM -0500, Stan Hoeppner wrote:

tomas@tuxteam.de put forth on 6/26/2010 11:52 PM:

...

The references are spot-on. The IDLE command is just designed to notify changes to the *selected* mailbox [...]

None of this is laid out in RFC2177.

Maybe not explicitly. But have you looked at 2177? The way it works is thus:

(1) client says SELECT <mailbox> (2) server says OK (among other things) (3) client says IDLE (4) server says + (meaning: I'm waiting for more)

(5) now connection is quiescent until:

(6) either server says EXISTS (or EXPUNGE)

(7) client ends IDLE with DONE

Note that at step 7 in this example scenario, server has *no way to say to which mailbox this EXISTS refers to*. It mus refer to the selected mailbox and just to that.

That means that the way IDLE is specified, only notofications referring to one mailbox can be received.

None of the relevant things we're discussing are in RFC2177 anyway.

See above: RFC2177 tells us: "one connection per mailbox", and...

                                                                They're

in RFC3501, which is rather lengthy.

...RFC3501 tries to fix exactly that.

Regardless, my point is valid and stands: there is no (good) reason for the protocol to require multiple socket connections when everything can be accomplished more efficiently (in terms of resources consumed) over a single socket.

Well, that's the point of IMAP NOTIFY in RC3501. If this extension is implemented, one socket will suffice to receive notifications concerning several mailboxes. As long as we don't have that, clients will work around the limitations of NOTIFY by opening several connections. That's what Patrick was saying upthread:

"Or even better: IMAP NOTIFY [2] gets implemented in dovecot and in Thunderbird, and one TCP connection suffices for an arbitrary amount of "push" folders :)"

    I'm sure many people more qualified than me have pointed this out WRT

the IMAP protocol over the years.

I don't exactly know what you mean by "this". Before the IDLE extension, there was no (portable) way to get notifications via IMAP at all: clients had to poll. After the IDLE extension, there's just one mailbox per connection (no way to argue around that: the protocol is just designed this way) the client gets nudged. As this was seen as a limitation, now there's IMAP NOTIFY, which just does what you ordered. Now to convince client and server implementors to use it :-)

Regards

• -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFMJ1mMBcgs9XrR2kYRAs0JAJ0b7vmolytQgw8xAk/lLO+HZ5SqdgCcCh0D wjqLKE/nWpTzfvcjpIh/OLM= =p4Jd -----END PGP SIGNATURE-----

Charles Marcus put forth on 6/27/2010 11:37 AM:

I think what matters is that the number of connections that TB is expecting to be able to use be *less* than the max supported by the IMAP server. I do know that as soon as I changed the MAX number of connections per IP in Courier to 5, all of our problems went away.

When you get a chance fire up top and take a look at the CPU time used by each of the 5 imap processes associated with a given TB user. Four will be at zero or maybe one or 2 seconds. You'll see the vast majority of the CPU time is on the fifth imap process.

This is exactly why I limit TB to one connection. My mildly educated guess is that the other 4 connections are being used strictly for IDLE processing, if at all. For me the additional 4 unused or rarely used connections isn't worth the additional overhead, even though said overhead isn't terribly high. Having 250 imap processes running on the system for 50 logged in users is something I just can't stand, given that 50 imap processes have no trouble servicing those same 50 users.

Hopefully NOTIFY fixes all of this, and hopefully TB and Dovecot will support it before too long. As is, I'm more than happy with single connections and polling, and one imap process per user. The big downside to my setup? Users might have to wait up to 60 seconds for a new email notification. However, they don't know they waited 60 seconds, so what does it really matter? As far as they know it just now arrived.

-- Stan

On 6/28/2010 11:58 AM, Charles Marcus wrote:

Hmmm.. care to share a good set of command-line switches for top that will let me see per user processes like that? Just running 'top', I only see the active processes, and there are really very few...

top -U<username>

-- Dave Brenner - david@toledotel.com The Toledo Telephone Company, Inc.

Charles Marcus put forth on 6/28/2010 4:19 PM:

On 2010-06-28 3:30 PM, Dave Brenner wrote:

...

On 6/28/2010 11:58 AM, Charles Marcus wrote:

...

Hmmm.. care to share a good set of command-line switches for top that will let me see per user processes like that? Just running 'top', I only see the active processes, and there are really very few...

...

top -U<username>

Doesn't work to show just a single users connections because all imap processes run as same user.

I guess this is different with virtual users than with system users? Are you using virtual or system users Charles?

-- Stan

Charles Marcus put forth on 6/29/2010 6:28 AM:

On 2010-06-28 9:05 PM, Stan Hoeppner wrote:

...

Charles Marcus put forth on 6/28/2010 4:19 PM:

...

On 2010-06-28 3:30 PM, Dave Brenner wrote:

...

On 6/28/2010 11:58 AM, Charles Marcus wrote:

...

Hmmm.. care to share a good set of command-line switches for top that will let me see per user processes like that? Just running 'top', I only see the active processes, and there are really very few...

...

...

...

top -U<username>

...

...

Doesn't work to show just a single users connections because all imap processes run as same user.

...

I guess this is different with virtual users than with system users? Are you using virtual or system users Charles?

Virtual of course... doesn't everyone? ;)

I dare to be different. :)

-- Stan

On Tue, Jun 29, 2010 at 16:16, /dev/rob0 <rob0@gmx.co.uk> wrote:

On Tue, Jun 29, 2010 at 07:28:52AM -0400, Charles Marcus wrote:

...

On 2010-06-28 9:05 PM, Stan Hoeppner wrote:

...

I guess this is different with virtual users than with system users?  Are you using virtual or system users Charles?

Virtual of course... doesn't everyone? ;)

Virtual mailboxes have their place, of course, but they're overused, especially at small sites. I suppose this might be in part because most HOWTOs are for virtual.

I recently saw someone asking for help, having set up a "simple" server with virtual mailbox (yes, singular) and mysql! The querent was trying to add a SECOND account and did not know how!

And what do the MySQL proponents say about that?

I started into mail on a very small scale, and that approach served me well. I set up Postfix by reading the comments in main.cf; later when I got the idea that I might want POP3 or IMAP, I uncommented lines in inetd.conf (popa3d I think, and uw-imap), and they worked. When kids got old enough to use email, adduser[1] and there they go.

It's nice you had that. Most of the mail servers I did in the past didn't even have POP (users logged into a shell account to read mail). Only recently did I even get into IMAP. IMAP was new to me, as was Dovecot (obviously). Not so with Postfix (or Sendmail, for that matter ... but I won't go back there). Oh, and I tried Qmail for a short stint.

I didn't get into virtual mailboxes until later, on a job, and when I did, I knew enough to question the wisdom of it. Why did we need this additional authentication database? All our users were using Samba via system accounts too. It could have been all integrated! The "advantages" I was told of doing it the virtual way were all based on misunderstandings. (One common one: "I don't want mail users to have shell access." Giving them a shell of /bin/false and/or setting sshd_config(5) access controls does the job.)

If there is one domain, and each user has an email name matching shell names, that's fine. Use system accounts and shells of /bin/false or whatever. But once you have more than one domain, it is possible to have collisions. This can happen with company mergers. User "jsmith@companya.com" and "jsmith@companyb.com" could be two different people who need to continue working with their original email addresses, while the former companies operate as business units under a single merged mail server.

There are two (or more) different kinds of virtual, too. One involves mapping multiple users of different domains into distinct system usernames which are not necessarily the same as the LHS of their email address. Now a mapping has to be made, and IMAP logins aren't as straight forward for users (one user logs in as "jsmith" and the other logs in as "jsmith2" ... and what if the 2nd J. Smith is the one that takes the reins as CEO).

The other is usually called virtual, but I personally don't, since I consider it to be real. I have:

mail_location = maildir:/home/mail/%Ld/%Ln/mail

I don't see that as any more or less virtual than where every user has a shell account and the config reads:

mail_location = maildir:/var/spool/maildir/%Ln

I don't think of that as virtual because the user names and domains are unchanged (I'm now counting lower casing the names).

I think many if not most of the questions we see on these lists are from people who have made a bad choice of using virtual mailboxes, often as a direct consequence of that choice.

Are you referring to all kinds of virtual? Or just some? Which sets of terminology are you using?

Personally, I consider it a bad choice when email addresses are mapped to system users, where LHS doesn't always match the shell user name. I consider it bad because of the confusing maintenance involved. The other two methods (username@justonedomain with mailboxes literally owned in the filesystem by the user ... or the way I do it now with multiple domains and the mailboxes literally owned in the filesystem by a designated role system user) I consider to be OK.

Email grew up with Unix, so it's no accident that Unix shell usage has very nice integration with email. Probably a lot of the folks reading this list would not even need an IMAPd if they knew more about these things.

And it also grew up working with either one domain, or multiple domains having a completely joint user set.

But mail can also function just fine when the MAIL USERS are completely isolated from the SYSTEM USERS. That doesn't mean doing this makes sense for everyone. But it can make sense for many (multiple domains and disjoint username sets).

I often encounter frustrated newbies who tried to do the whole thing all at once. It makes much more sense to start off small, throw in the relational databases later, learning the finer points of how to manage your OS along the way. The secret is that you can have a fully-functional mail server with very little bother, using system accounts. Postfix (or other MTA) and Dovecot will pretty much Just Work, right out of the box.

For learning mail servers, that might make sense. For setting up a mail system for a business with multiple disjoint domains (merged businesses, ISPs), it usually doesn't. OTOH, such a setup shouldn't be done by a beginner, either.

[1] adduser is a Slackware-specific frontend wrapper script for useradd(8) and other tools from the shadow package.

And, Slackware is, IMHO, a better choice (or FreeBSD or OpenBSD ... Gentoo might be OK, too). I used Ubuntu on mine most recent one due to it being a complete business startup and many other servers were being set up at the same time and which server would do what hadn't been established. I've already begun the plans to migrate to Slackware64 for the mail server and web server, with the core service software (Apache, Dovecot, PHP, Pike, Postfix, Python, and a few others) built from the latest source.

On 2010-06-29 4:16 PM, /dev/rob0 wrote:

Virtual mailboxes have their place, of course, but they're overused, especially at small sites. I suppose this might be in part because most HOWTOs are for virtual.

That's just plain silly. Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch.

That said, there is nothing wrong with using system users, if those users also have/need shell access, but if they don't virtual users is just as easy/legitimate as system users with no shell access.

It's more a matter of the individuals skill level.

--

Best regards,

Charles

On 2010-06-30 9:03 PM, Stan Hoeppner wrote:

Charles Marcus put forth on 6/30/2010 5:11 PM:

...

On 2010-06-29 4:16 PM, /dev/rob0 wrote:

...

Virtual mailboxes have their place, of course, but they're overused, especially at small sites. I suppose this might be in part because most HOWTOs are for virtual.

...

That's just plain silly. Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch.

That said, there is nothing wrong with using system users, if those users also have/need shell access, but if they don't virtual users is just as easy/legitimate as system users with no shell access.

It's more a matter of the individuals skill level.

So exactly what does this say about the skill level of people who have implemented, and continue to implement, both solutions, Charles?

That they are most likely capable of determining for themselves if/when to use system user and when to use virtual users?

I don't get the question...

--

Best regards,

Charles

On 2010-07-01 1:04 PM, Stan Hoeppner wrote:

Charles Marcus put forth on 7/1/2010 6:39 AM:

...

On 2010-06-30 9:03 PM, Stan Hoeppner wrote:

...

Charles Marcus put forth on 6/30/2010 5:11 PM:

...

Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch.

That said, there is nothing wrong with using system users, if those users also have/need shell access, but if they don't virtual users is just as easy/legitimate as system users with no shell access.

It's more a matter of the individuals skill level.

...

...

So exactly what does this say about the skill level of people who have implemented, and continue to implement, both solutions, Charles?

...

That they are most likely capable of determining for themselves if/when to use system user and when to use virtual users?

I don't get the question...

Apparently you did get the question because you answered it correctly. However, your answer contradicts your "skill level" assertion above.

No... my comment was simply offhand, and not intended to be exhaustively comprehensive, and you decided to pick nits...

How about:

"It's more a matter of the individuals skill level, what they are used to, their specific need(s) for the specific situation, what some PHB may think is needed, and how much leeway said PHB gives you."

There are probably other conditions, so feel free to insert whatever else you feel may 'complete' it to your satisfaction... ;)

--

Best regards,

Charles

On Thu, Jul 1, 2010 at 02:28, Frank Cusack <frank@cusack.net> wrote:

On 6/30/10 6:11 PM -0400 Charles Marcus wrote:

...

That's just plain silly. Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch.

I agree. I am always in favor of virtual users, it just gives you a lot more flexibility. I find system users MORE complicated to setup, actually. You have to worry about system security in addition to IMAP stuff.  You always have to refactor things down the road and starting off with system users just makes it more unpleasant.

I find a system-user scheme more complicated only when there is not a one-to-one relationship between the system user base and the usernames in one domain. I tend to use a non-system-user scheme more, now, because of things like having different sets of users in different domains, where, if not now, possibly in the future, a LHS will conflict with a system user, meaning I have to map the relationships. In cases where there is one domain and LHS will be the same as the system user forever (about 3 to 5 years in internet time), I'll use system users (with role accounts either forwarded or as real system users, depending on need). Otherwise, the multi-domain, multi-user-set, all stored under one system user, scheme (that I don't like to call virtual because there is nothing virtual about it once you avoid thinking in terms of system users) works quite well. A hybrid, where one or more domains are designated for system users, could still coexist with the multi-domain scheme.

On Thu, 1 Jul 2010, Noel Butler wrote:

(I wrote a script to convert from vpopmail structure to a better structure when we moved from that mess to postfix/dovecot/mysql a few years back, that conversion, including moving mail took all of 45 minutes, most of that was copying mail, in the early days I did not like nor trust postfix, but are with it today and wouldnt use anything else again, in case I change jobs I've always kept my converting script hehe)

Sounds like something to publish on the Dovecot wiki. :)

(says the guy who's supposed to do a vpopmail conversion)

C

Hrmm., boy, so far OT now I'll finish...

So, my recommendation, is to plan for what might be some day, rather than wait until that "someday" arrives.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Wed, 30 Jun 2010, Charles Marcus wrote:

On 2010-06-29 4:16 PM, /dev/rob0 wrote:

...

Virtual mailboxes have their place, of course, but they're overused, especially at small sites. I suppose this might be in part because most HOWTOs are for virtual.

That's just plain silly. Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch.

Hmm, I understood Rob's post arguing that almost every Unix daemon "just plainly works" with system users. And, IMO, this is true for both: MTA and Dovecot. The requirements are low, because you have system tools to create users, installed daemons are pre-packaged to use them. Install, and you are set.

I do _not_ argue about security here. I really wonder why some distros still allow ssh-access by default for every user and some don't. Even a virtual-user based setup requires system users, so one cannot ignore uid related security either.

I also don't argue about flexibility.

Rob is talking about a newbie setup (IMHO) and I do agree to him. Once one got accustomed to the field of mail-related services, one can make decisions.

It's more a matter of the individuals skill level.

Well, a "system user" setup requires almost no skill of mail-related stuff ;-)

Regards,

Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCxK27+Vh58GPL/cAQKcfAgAkhTpfP+VIrWhreopLsULoqV5dyFCy3gd +Tx+BnKfy3or/nHjke0sSVzdf6O6NUuv5TW33d9vKSXGNXhQz4A7XtqxaU3K6Ze1 hm9gFYAfPNtSGEe1v8d+rxnugYmDfW8NV+03Wx0qRM2bmFZeYZQOFztRCpsIcAe8 DHMUCCWaJ2DZMc6LqxssripgwW9H8rIyiBWKbWyduqkuF52S07BL+RPJPzRfBgZc vnF0vFE8SiDVsp6kc3ofW86Mm8FS/efQEXyqomeafdzyScrZZg4gisXECNrcJTey luKuhgAZa7bwkKZi91xpf+zoI8UQghk5vmoGocL++9UjJafju35NZQ== =Q5PF -----END PGP SIGNATURE-----

On Thu, 2010-07-01 at 04:01 -0500, Stan Hoeppner wrote:

Anyone who isn't looking at mail logs or log summaries daily and taking action on any problems needing attention doesn't count as a mail OP.

That's one of the most ridiculous things I've seen todate. Do you seriously expect ISP admins that may have for instance, 16 front end SMTP servers, each processing around 1.4 million connects a day, and accepting around 900K msgs each a day, are going to seriously sift through each servers logs every day?

I don't think thats going to happen anytime soon

On Thu, 2010-07-01 at 06:14 -0400, Jerry wrote:

I agree. If the system is constructed correctly it certainly does not need that sort of attention. There is software available that can monitor the system to a high degree of satisfaction. However, Noel, I firmly believe that there are OPs (SAs ?) that greatly exaggerate the degree of difficulty of their job. I guess we all like to feel we are indispensable.

I'm certain that's the case, anything setup correctly, you should be able to walk away and almost forget about it, the only thing to do is modify anti spam rules to catch variants of new spam, all of 1 mins work, tops, the rest of the time is helping manage the rest of the network :)

Mail Administration is not complicated, all too many people like to over complicate their setups and only cause themselves work.

I've had more than one CEO in the past say to me that they like to see key NOC staff doing nothing, because it says to them the network is working perfectly.

All too many do not automate things or write scripts/cron tasks, complicate their network and tinker, because as you said, they need to feel indispensable, if only their managers had a clue.

On Thu, 01 Jul 2010 12:12:37 -0500 Stan Hoeppner <stan@hardwarefreak.com> articulated:

I'd just get a huge kick out of cross posting what the two of you state here to spam-l and watching you get eaten alive due to this "runs itself if setup right" hands off management approach to email systems. Rich would send you home with your tails between your legs like little scared puppies. Neither of you sub there so it wouldn't do any good. T'would be very entertaining if you did though.

Here we go; no longer can you justify your position so now you attempt to change the focus of it, and/or attach the responders of your post.

I stand by my assertion that a properly configured system basically runs itself. Software updates, etc do on occasion require direct intervention by the system maintainer; however, if I have to reconfigure the system on a daily basis it is more than obvious that I have failed to properly set it up to begin with. In virtually every case when a serious problem has arose on the system, it could be directly tied to the "PEBKC" principal.

By the way, I have no knowledge of this "Rich" individual, nor do I give a F**K either. Obviously you are mesmerized by, and perhaps even sexually attacked to him, so I suggest that you consult him from now on when a problem arises.

-- Jerry ✌ Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

The Israelis are the Doberman pinschers of the Middle East. They treat the Arabs like postmen.

Franklyn Ajaye

Noel Butler put forth on 7/1/2010 4:46 PM:

< snipped the juvenile stabs >

oh but as a parting shot, with all that mail we get, little spam, scams or viruses gets to our users, that says we are doing something right, and it hasn't been since around 2004 that we had any particular smtp server in an DNSBL, and then it was only one of a dozen (0 day virus infected windows weenie) , and although I was once a member of the "inner boys club" being spam-l, Jerr'ys comment and my agreeance are

You've been looking at this from the wrong perspective the entire time, and apparently completely missed my original point, which was keeping a close eye on what's going on with one's SMTP servers.

You mentioned nothing of outbound mail in your diatribe, only inbound. That means you only perform half of your duties as a mail OP. There are numerous scenarios in which outbound mail will get deferred, sometimes for up to 5 days or more. Users have no clue there is a problem unless the receiving party is expecting the particular email and it doesn't arrive in a timely manner. By your own statement it would appear that you simply wait until the deferment times out and your user finally receives an NDR.

A good seasoned mail OP is going to monitor his/her logs, via any number of methods, and when a deferral problem arises, investigate. If the cause of the problem is on the other end, said OP will attempt to contact the postmaster and work with him or her to resolve the problem.

even more applicable to them, it totally amazes me how many SA's get away with this 'self justification' of their employment, again., if only their employers really knew.

At many organizations email is a critical communications tool and is relied upon just as a telephone is (whether relying on email is smart of not will continue to be debated for eons). These organizations want and need proactive mail OPs, ones who will take initiative and begin solving problems such as that mentioned _before_ users even know there is a problem.

To answer your question, yes, their employers _DO_ "really know" and that's exactly why they hired them. They want proactive postmasters and SAs. Most businesses and large organizations do, or at least the ones who can afford a decent staff. For the small/medium business with a one man IT shop or a small staff where everyone wears many hats all day long, this isn't feasible. But those with a real operations staff, most want the type of postmaster or SA I've described. They _don't_ want the type who sits around waiting for users to report problems. Preferably they want the problems solved proactively so their users never know there was a problem.

-- Stan

On Fri, 02 Jul 2010 11:11:12 -0400 Charles Marcus <CMarcus@Media-Brokers.com> articulated:

# postconf -n | grep delay_warning delay_warning_time = 15m #

Other than that I agree absolutely with the rest, except to note that most of this monitoring can be done automatically with tools designed to *watch* for warning signs, and this *may* have been what Noel was silently referring to...

I don't speak for Noel; however, that is precisely what I was referring to. There are numerous tools available to monitor system functions, mail systems, etc. The concept of having to review potentially thousands of pages of data every day is to maintain a mail system is unfathomable. If the senior mail system maintainer is discovering huge numbers of messages stuck in queue on a virtually daily basis it would indicate that something is not configured correctly. Yes, things do go wrong. However, if they are going wrong as a routine event then something else is the root cause. Usually, discovering the source of that problem is no more difficult than looking into a mirror.

People tend to exaggerate the difficulty of their job to justify its or their existence. There is really only one truly difficult job and that is a highway flag man. I know it to be true because after observing thousands of them in my time, no one can do it correctly.

Now, before Stan gets his knickers in a knot, I am not implying that the job of maintaining a system is not essential. Obviously it is. However, it is not rocket science or brain surgery. Yes, it takes training and dedication. The problem is that way too many individuals develop rotator cuff problems from patting them selves on the back for doing a routine job. Or to put it in the vernacular, "Get over yourself."

In any case, I am out of here. This thread has nothing to do with Dovecot, Thunderbird or virtual mailboxes (thanks to whoever hijacked the tread and changed the subject.)

-- 

Jerry ✌ Dovecot.user@seibercom.net

Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.

Satyrs have more faun.

Charles Marcus put forth on 7/2/2010 10:11 AM:

# postconf -n | grep delay_warning delay_warning_time = 15m #

That's disabled by default:

delay_warning_time (default: 0h)

The time after which the sender receives the message headers of mail that

is still queued.

To enable this feature, specify a non-zero time value (an integral value

plus an optional one-letter suffix that specifies the time unit).

Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). The

default time unit is h (hours).

It may work for some folks. A daily or twice daily error summary would probably be more useful to most SAs IMHO. Recall I stated something like "timely" not "immediate" response. ;)

Other than that I agree absolutely with the rest, except to note that most of this monitoring can be done automatically with tools designed to *watch* for warning signs, and this *may* have been what Noel was silently referring to...

Of course people use all kinda of automated tools to get this information, as they should. The "how" (method/tool) hasn't been part of this discussion/argument. Though, IIRC, he was making the argument that servers configured properly "run themselves" and thus require very little if any monitoring by an OP or SA, and if they did require such, the OP sucks because he didn't set the system up right in the first place. His entire statement regarding managing his mail system revolved around updating anti spam info, not dealing with delivery or other problems not related to spam.

-- Stan

On 2.7.2010, at 23.52, Charles Marcus wrote:

...

A daily or twice daily error summary would probably be more useful to most SAs IMHO.

It would be useful, yes, and I'd love to see this implemented. In fact this has come up on list more than once, and I seem to recall that Wietse has no interest in implementing it...

This reminds me: Any errors that Dovecot logs are bugs. I think a lot of people are ignoring and not reporting those.

On 2010-07-03 11:09 PM, Frank Cusack wrote:

On 7/2/10 6:52 PM -0400 Charles Marcus wrote:

...

# postconf -n | grep delay_warning delay_warning_time = 15m # ... It works for everyone who enables it. What the user *does* with the warning is their problem. I don't sympathize with idiots.

Wow, you must hate your job then, if you are an SA.

Not at all... I love my job, but one has nothing to do with the other. Indeed, if I *did* sympathize with idiots, *then* I'd probably be miserable in my job...

delay_warning_time is awful for a site of any size above "small".

I totally disagree - unless, of course, you have a lot of problems with delayed mail, in which case I would say that the SA has a problem that needs attention.

If nothing else, it is a good indicator of a clogged queue, which is an indicator of, again, a problem that needs attention.

As far as I can determine, even for a fairly busy server, unless you are a spammer/mass emailer, you should rarely if ever have any given email in the queue for more than a few minutes... mine rarely stay there for more than a second or two...

You just get confused users / complainers.

I have had a grand total of about 3 complaints in the last 3 or so years. Yes, this is a small company (about 50 users), and our volume of email is probably just average, so as always ymmv...

--

Best regards,

Charles

On 7/4/2010 2:08 PM, Stan Hoeppner wrote:

Charles Marcus put forth on 7/4/2010 12:57 PM:

...

in the queue for more than a few minutes... mine rarely stay there for more than a second or two...

With the popularity of greylisting these days I would think you'd be seeing at least a handful a day that sit in the queue for multiple minutes. That is, of course, unless your users never send to a new address/domain, merely communicating with already established relationships.

I assume -with all that THAT implies - that there is more involved in server configuration than just one parameter. Such as adding a second parameter enabling recipient verification - which could lead us to another discussion so I won't mention it and please forget I said anything - did anybody hear that-oh-look-Timo-just-released-a-new-version, yay Timo!

I would respectfully suggest we're getting just a little off-topic here

• can we confine discussions on this list to something Dovecot-related?

  Daniel

On Fri, 2010-07-02 at 09:26 -0500, Stan Hoeppner wrote:

Noel Butler put forth on 7/1/2010 4:46 PM:

< snipped the juvenile stabs >

...

oh but as a parting shot, with all that mail we get, little spam, scams or viruses gets to our users, that says we are doing something right, and it hasn't been since around 2004 that we had any particular smtp server in an DNSBL, and then it was only one of a dozen (0 day virus infected windows weenie) , and although I was once a member of the "inner boys club" being spam-l, Jerr'ys comment and my agreeance are

You've been looking at this from the wrong perspective the entire time, and apparently completely missed my original point, which was keeping a close eye on what's going on with one's SMTP servers.

*sigh*

are you really this stupid or just trolling ?

You mentioned nothing of outbound mail in your diatribe, only inbound. That

I guess your trolling childish mind over looked the comment about RBL, which I think kinda infers "outbound"

means you only perform half of your duties as a mail OP. There are numerous scenarios in which outbound mail will get deferred, sometimes for up to 5 days or more. Users have no clue there is a problem unless the receiving party is expecting the particular email and it doesn't arrive in a timely manner. By your own statement it would appear that you simply wait until the deferment times out and your user finally receives an NDR.

fuck me dead, if you think I am going to sift through all deferred log messages you seriously are not living in the same universe as I. you have NO idea on the volume of mail the servers I'm responsible for process, or my servers configurations or automated monitoring, so stop making your dumbass assumptions.

A good seasoned mail OP is going to monitor his/her logs, via any number of

sure, if there are a tiny SOHO like you or your other little spam-l mates. so sorry that we dont do what you do, but hey I guess the fact we get on average, 2 abuse complaints and maybe 4 or 5 general mail complaints from our users (unrelated to spam) a week, shows we know what we are doing, and given the volume of mail, I'll tell you now i'd still be over the moon if we got 20 abuse and general complaints a day! but the fact we dont, and it all runs smoothly, shows we know what we are doing. Pretty clear your capabilities are not up to the standard that I expect.

To answer your question, yes, their employers _DO_ "really know" and that's exactly why they hired them. They want proactive postmasters and SAs. Most

If you, or anyone on my staff wasted their time doing things like this I d sack your time ass in an instant, the only "looking at logs" that goes on here, is immediately after a software upgrade to ensure things are working.

now, be gone, I have nothing further to discuss with you troll, this thread left Dovecot topic a long time ago.

On Thu, 2010-07-08 at 08:42 +0200, Patrick Ben Koetter wrote:

• Noel Butler <dovecot@dovecot.org>:

...

*sigh*

are you really this stupid or just trolling ?

Seriously, I think you should all go offlist with your insults. Stop stealing other peoples attention with your dogmatic positions. Obviously you seem to have opposite positions and all of you seem to have a strong opinion why you take them. Stop trying to persuade the other to adopt your position. Accept that there are people who are different.

I am asking the list operator to close this thread.

p@rick

Ahhh, so your quite happy to continue with diatribe on-list though? I see, yes, I see exactly where you come from. Most people ceased reading this thread a long time anyway., Also, I aint trying to pursuade anyone to do anything, it be them who seek to change me, not that I care whatr they, or you think

On 7/1/10 9:59 AM +0200 Steffen Kaiser wrote:

I do _not_ argue about security here. I really wonder why some distros still allow ssh-access by default for every user and some don't. Even a virtual-user based setup requires system users, so one cannot ignore uid related security either.

huh? no virtual user system i've ever setup, or could conceive of, requires system users (above and beyond what the mail system inherently requires, of course).

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On Mon, 28 Jun 2010, Charles Marcus wrote:

With ps aux, I can see all connections, but they are all pretty much at zero with the exception of those where people are moving messages around and such...

ps xua|perl -e 'while(<>) { @a=split(/\s+/, $_); if($a[10] eq "imap") { $top{substr($a[11],1)} += $a[2] } } foreach $uid (sort {$top{$a} <=> $top{$b} } keys %top) { print $uid, " ", $top{$uid}, "\n" } '

The idea is to catch the lines with "... imap [uid ip]"

To reverse the sort swap $a and $b in sort { }, so one can combine the script with the commands head and watch, in order to build a poor man's top ;-)

To include pop in the stats, add

|| $a[10] eq 'pop3'

to the if(). Well, maybe your output of ps differs as well.

Regards,

Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCmkub+Vh58GPL/cAQLRZwgAzyp6RuvwBzAsdMf3PogaBzucPf9Ps2ST gDriAp2HvDJs3gz65oMWSGGB0iEEeo0JUJbEUk7y10/nHv89P4pPecb9mmrSq59f 6SJcsw01Bpzc3W76LQV9HXYix1ybr8QTpMEQHlk5Hr1Hkwp0488PdZ5IPSnW1Cxh qhyWqOFCp7j4MQ+VGk7Rp0heFWqTw9DaykceP6fMAz/NWVwZvnbTQ/XdsEruGKev 7okXLgO6NN38cQfV/RsX5SpJ3+Yqx4Hx7o3Quut7HtNybG/hJeqBXahRMg56SIkA /RkFmr87BXDU+2srb6rtRxk/EBy5Qekb4R3OMZkRno9amRdUdpLV6Q== =S4/r -----END PGP SIGNATURE-----

Brent Bloxam put forth on 6/29/2010 3:36 PM:

Thunderbird is a modern threaded application, users are able to perform many parallel actions. The IMAP protocol returns data for one action at a time, so in order to follow through with the user requests, it delegates commands to multiple connections. This may not be apparent when dealing with mail folders with few messages that have actions completing in a few seconds, but when dealing with large amounts of data the need for multiple connections becomes apparent (unless you're a patient person ;)

So you're saying that having multiple connections gives TBird more bandwidth to the server?

-- Stan

Thanks for your *complete* answer Charles .

On 06/25/2010 02:32 PM, Charles Marcus wrote:

On 2010-06-25 7:48 AM, Frank Bonnet wrote:

...

Some users had occasionally a problem with Thunderbird , some random emails with attachement(s) cannot be read anymore the email appears empty and TB seems to enter in an infinite loop saying it is downloading the message

The only solution I found was :

1 - stop thunderbird on the client machine 2 - remove the .imap user's directory 3 - restart thunderbird on the client machine

After doing that all emails re-appear normally and attachements are readable/downloadable again.

ALL our clients use the IMAP(S) protocol.

Anyone had the same problem ?

Yes, this is a known bug that is fixed in 3.1...

An easier fix is to simply drag the problem message to a different folder - you can usually open the attachment right away, but often only once before the local copy gets corrupted again. I created a _Temp folder for our users having this problem. Sometimes you'd have to rebuild the index on that folder, even more than once. Then you could move the message back to the Inbox (or wherever it was).

I seemed to notice this was mostly a problem with messages sent from Outlook clients.

Anyway - update to 3.1 and the problem is gone.