How big is a BFS directory when it starts to be a sub-optimal performance problem? Access time, memory, startup, whatever anyone has noticed.
Recent discussion of attributes here reminded me of a pile of emails I strored up from a former email provider. I copied a few hundred over and set them up with MAIL: attributes, and it seems like a fine idea. I imagine I’d put them on a USB device. So that would be, I don’t know, like 20,000, 30,000 files, all with email attributes. Quite static, there’d be no subsequent change.
All in one directory? Split it up a little? A lot?
Back in the '70s, I had the notion to learn to read Chinese, but I quit when I realized that there are mulitiple ways to write it and my dictionary and textbook were mutually unintelligible. Is there a demonstration in there somewhere of how many files there are in one large directory, and tis relative performance characteristics?
I went ahead with it, and see no problems here with 26,000 emails in a single directory. (I rebooted to preclude the possibility of a cache.)
As I was moving the files out of their separate directories, the renames seemed to slow down somewhat. Total elapsed time was about 20 minutes. Data ca. 4.5 Gb.
There are no built-in restrictions. An index is just a directory, too, and it can hold hundreds of thousands of files without anyone noticing.
In any case, file action performance mostly depends on the number of indexed attributes. Your mileage might vary anyway…
It turns out that the attributes weren’t indexed on this USB disk, so I added indices for MAIL:name, MAIL:from, MAIL:subject and MAIL:when. Via fs_create_index(), and fs_read_index_dir() shows them.
But something is still missing, because “Find …” allows only Name/Size/Modified attributes, even for /boot. So at the moment I’m not sure the existing attributes would be indexed when the index dir was created afterwards (probably not, eh?), nor am I sure indices would be any use here anyway.
I don’t know, that doesn’t seem to help. Hard to say for sure - the command line usage is odd.
usage: reindex [-rvf] attr <list of filenames and/or directories>
-r enter directories recursively
-v verbose output
-f create/update all indices from the source volume,
"attr" is the path to the source volume
“attr” is the path to the source volume? That’s odd. Anyway, no joy from trying it. lsindex shows the MAIL: indices I created yesterday.
When using -f. That is, reindex -rf /template_volume ~/some_dir would create and update indices for the all the attributes defined in template_volume for all the files in ~/some_dir.
In your case I’d say you just want reindex -r 'MAIL:*' your_mail_directory
Thanks! that did it. It took a few minutes, and I think it might have gone smoother if Tracker hadn’t had the directory open while I did it, but it’s working fine.
There is a little lag - I mean, when I open that 26,000 entry directory, on a USB stick disk, Tracker is a couple of seconds getting all of them onto the window. Which I would totally expect.
