Massive /tmp files
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
/email/domain/username/.Folder/tmp
The files in there are normal maildir email files.
I think the dir ran out of inode space so its not ongoing problem. My question is only can I purge that folder without breaking something more? If it tries to re-fill again I will try to debug
Thankyou!!
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
I've had some issues (in the past) with thousands of links being created to each message, resulting in many millions of "files". I even wrote a script to remove all the linked files leaving only one actual file behind, but I haven't had to do that in quite a while (a year? More?).
I don't seem to have the script handy (it's around somewhere), but this is basically what I did:
https://unix.stackexchange.com/a/286780/61289
I think the dir ran out of inode space so its not ongoing problem. My question is only can I purge that folder without breaking something more? If it tries to re-fill again I will try to debug
AIUI tmp is temp and can go away, though I'd stop dovecot first just to be extra safe.
-- Science is the foot that kicks magic square in the nuts.
On Jan 22, 2021, at 8:02 AM, @lbutlr kremels@kreme.com wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
How would you distinguish a hard link from a “real file”?
rg
On 22 Jan 2021, at 09:07, Ron Garret (gmail) ron.garret@gmail.com wrote:
On Jan 22, 2021, at 8:02 AM, @lbutlr kremels@kreme.com wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
How would you distinguish a hard link from a “real file”?
ls -l will show the number of hard links to a file in the first column after the permissions (or it showed the number off files (including . and ..) inside a directory if it's a directory entry).
#v+ 10:58:53 :/ $ touch test.file 10:59:03 :/ $ ls -lsn total 0 0 -rw-r--r-- 1 502 0 0 Jan 22 10:59 test.file 10:59:09 :/ $ ln test.file mynewfile 10:59:19 :/ $ ls -lsn total 0 -rw-r--r-- 2 502 0 0 Jan 22 10:59 mynewfile -rw-r--r-- 2 502 0 0 Jan 22 10:59 test.file ^ number of hard links 11:04:18 :/ $ ls -lnd . drwxr-xr-x 4 502 0 128 Jan 22 10:59 /tmp/test/. ^ number of files #v-
There are other ways, including the stat command.
-- Rent a flat above a shop, cut your hair and get a job, smoke some fags and play some pool, pretend you never went to school and still you'll never get it right cuz when you're lay'n in bed at night watching roaches climb the wall if you called your dad he could stop it all.
On Jan 22, 2021, at 10:05 AM, @lbutlr kremels@kreme.com wrote:
On 22 Jan 2021, at 09:07, Ron Garret (gmail) ron.garret@gmail.com wrote:
On Jan 22, 2021, at 8:02 AM, @lbutlr kremels@kreme.com wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
How would you distinguish a hard link from a “real file”?
ls -l will show the number of hard links to a file in the first column after the permissions (or it showed the number off files (including . and ..) inside a directory if it's a directory entry).
Ah, I misinterpreted the question then. You meant (I presume) “Are they 8 million distinct files, or 8 million hard links to a (much) smaller number of actual underlying inodes.”
So then my next question is (and I’m not intending to challenge you here, I’m just trying to get a better understanding of how dovecot works under the hood): where would these hard links come from? What does dovecot use hard links for?
rg
On 22/01/2021 20:29 Ron Garret ron@flownet.com wrote:
On Jan 22, 2021, at 10:05 AM, @lbutlr kremels@kreme.com wrote:
On 22 Jan 2021, at 09:07, Ron Garret (gmail) ron.garret@gmail.com wrote:
On Jan 22, 2021, at 8:02 AM, @lbutlr kremels@kreme.com wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
How would you distinguish a hard link from a “real file”?
ls -l will show the number of hard links to a file in the first column after the permissions (or it showed the number off files (including . and ..) inside a directory if it's a directory entry).
Ah, I misinterpreted the question then. You meant (I presume) “Are they 8 million distinct files, or 8 million hard links to a (much) smaller number of actual underlying inodes.”
So then my next question is (and I’m not intending to challenge you here, I’m just trying to get a better understanding of how dovecot works under the hood): where would these hard links come from? What does dovecot use hard links for?
rg
There should not be any files under /tmp normally. If there are, you can look at what they are, and decide whether to rm them or move them under /new.
Maildir delivery says that you first write the file under /tmp and then link-unlink the file into /new.
Aki
On 22 Jan 2021, at 11:29, Ron Garret ron@flownet.com wrote:
So then my next question is (and I’m not intending to challenge you here, I’m just trying to get a better understanding of how dovecot works under the hood): where would these hard links come from? What does dovecot use hard links for?
I have no idea. As I said, it happened to me which is why I mentioned it as a possibility. My fix was to remove all the linked files and leave the originals. I did this a few times and then the problem went away at some point.
This was the initial post, it was about links int eh Hunj folder and I suspected sieve might be the cause
Message-Id: 79A39D36-EDC5-40CA-B187-F45C6623F0C1@kreme.com
-- Spontaneity has its time and place.
On Jan 22, 2021, at 8:02 AM, @lbutlr kremels@kreme.com wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
How would you distinguish a hard link from a “real file”?
rg
On 2021-01-22 16:02, @lbutlr wrote:
On 21 Jan 2021, at 18:08, MRob mrobti@insiberia.net wrote:
Hello, I just found user who has single folder (standard maildir format) that filled with over 8mil files and 800GB in the /tmp subdirectory of that folder:
Are they real files or hard links?
They are files with no other links.
I've had some issues (in the past) with thousands of links being created to each message, resulting in many millions of "files". I even wrote a script to remove all the linked files leaving only one actual file behind, but I haven't had to do that in quite a while (a year? More?).
I don't seem to have the script handy (it's around somewhere), but this is basically what I did:
https://unix.stackexchange.com/a/286780/61289
I think the dir ran out of inode space so its not ongoing problem. My question is only can I purge that folder without breaking something more? If it tries to re-fill again I will try to debug
AIUI tmp is temp and can go away, though I'd stop dovecot first just to be extra safe.
Date on files is over 2yrs ago with dates all newer than files in "cur" folder so looks like user was moving massive number messages then dovecot choked? But 8mil seem impossible for almost any user. I can't read all of them :) but looked at a few and they are normal messages, tho a lot seems spam.
Will have to talk to the user but I'm inclined to purge.
Thank you for helping me!
participants (5)
-
＠lbutlr
-
Aki Tuomi
-
MRob
-
Ron Garret
-
Ron Garret (gmail)