I’ve finally had it with my mail software. I need advice on what I can do.
Here’s the situation: Mac OS X (that’s not going to change), the standard Mac Mail program, everything up-to-date with the latest versions. I’ve got about 20 folders set up in Mail, with filters to automatically redirect incoming mail to sensible places — student email gets top priority, for instance, a couple of listservs get their mail shuffled off to a convenient holding pen, mail from family members gets its own place, etc. Spam is currently not a problem; I’ve set up all my email accounts to forward through gmail, where the spam gets held up and eventually discarded, and I never even have to look at it.
I get a lot of email, even without spam. Every three months or so, I toss everything out of email and into a new OS folder just for archiving purposes, and reset everything to a mostly clean slate. It’s been a couple of months now, and I’ve got roughly 100,000 messages in Mail. I know. It’s unreal. Why all these people want to send me mail, I don’t know, but there it is.
Now here’s the problem: Mail can’t handle it. The software just gets slower and slower and s l o w e r, until, like now, it’s at the point where I get a message that I’ve got incoming mail, I click on the folder where it’s stored, and I can go fix a pot of coffee while Mail struggles and strains to just display the gorram list of messages. If I look at the processes, Mail is eating 80-90% of the CPU time for several minutes to show me one folder full of messages. And don’t tell me to just throw out all the accumulated mail: I’ve tried. I’m trying right now. I said to myself, OK, this folder with 10,000 messages in it? Goodbye. I selected all, hit delete, and waited — for two hours. Now it’s in Mail’s trash folder. I select “empty trash” and … wait another two hours.
Mac Mail is no good if you are trying to cope with a lot of email. It’s dead, dead, dead as far as I’m concerned. I’ve begun to actively hate Mail — it’s my daily enemy.
So someone tell me what nice, sensible GUI mail reader is out there for Mac OS X that is actually intelligent about managing large volumes. Gmail is nice, but I want something that will let me deal with mail offline.
Post it here, though. Please, please don’t email me suggestions, unless you really want to make me cry.
Before you give up on Mail, try this tip on Hawk Wings, an excellent blog devoted to Mail.app
Mail Steward, is an archiver for mail that helps deal with vast archives of mail.
However, in my experience, none of the major mail apps are very good at this sort of thing.
Gosh, PZ, I feel bad because I’ve probably sent you an email or two that wasn’t all that substantive, just general chattiness. That would be adding to the burden you described, and if that’s the case, I apologize. No reply is needed, and I hope you find the solution you need.
Sincerely…SH
Doesn’t Google Desktop let you access your GMail messages offline? I just downloaded it last night for that reason, but haven’t had a chance to play with it at all yet.
Try Mailsmith, from Bare Bones software. It’s the closest thing to an industrial-strength mail program in existence on OSX.
Only downside is that there’s no Universal version if you happen to be on Intel.
Try rebuilding your mailboxes.
I don’t actually know if it will handle the load, but Thunderbird has worked very well for me for quite some time. But I don’t get anywhere near that level of mail, I’m running Windows, and it is from Mozilla (whose products do tend to slow down under heavy load). If you don’t figure out a better solution, Thunderbird is worth a shot.
No ideas myself but I just posted a request for help on the Applegeeks forum. I think if anybody would know and care to help, they will! Who knows, PZ might even show up as a cameo in the strip!
You coudl try the Mail Vacuum Automator workflow
http://automatorworld.com/archives/mail-vacuum/
well, you could skip the gui altogether and install mutt from fink or pkgsrc.
long-winded boring instructions available upon request.
Has anyone any experience with Magellan pro, which is advertized as being good for handling large amounts of email?
http://www.makienterprise.com/magellanpro/magellanpro.html
I don’t think this client can handle HTML. Is that a necessity?
Also- P.Z. do you have enough memory installed? Does anyone know if that could be a limiting factor?
Something is wrong. For comparison, I have 10,000 messages in my Inbox (1 GB), 7000 messages in my Sent box, and 9 “old mail” boxes ranging from 2600-6000 emails each. Moreover, they are all out on an IMAP server, which should slow things down even more. It does take Mail a few minutes to start up and sync with the server, but once it is going, everything is pretty snappy.
I did have a problem a while back when Mail got really pokey, and started chewing up huge amounts of processor time. I threw out all of the Mail associated files on my system and made Mail resync to the IMAP server, and everything has been fine since them.
You could also quit Mail.app (if you want to throw out an entire folder) and go to User —> Library –> Mail and then trash the contents of the mailbox you wan to whack.
I don’t think that would be my choice, though.
Mail is indeed like that, I’m sitting at a 2 X 2.0 G5 with 8 GB of RAM and this used to happen to me once in a while. I hit on the suboptimal regimen of deleting (archiving) mail much more often and the Automator Mail Vacuum Workflow
I haven’t tried this and can’t promise it’ll help, but a quick search of MacUpdate brought me to http://www.mailsteward.com/ MailSteward 7.7.1. A casual glance at its feature chart makes me think the less expensive Lite version may be all you need, but I dunno. Good luck.
Holy Lord, the thought of reading 100,000 messages in Mutt boggles the mind.
I’ve heard very good things about the tips mentioned in the first post, and about Eudora for OS X.
It does sound like there’s an underlying file problem going on and it may not neccesarily be the Mail program itself.
I use Lotus Notes at work… and I hate it.
I tried the mail vacuum script — it got the envelope file down from 77 to 66 MB. So far, no performance improvement at all. Unfortunately, it’s still also slowly cranky through flushing out the trash from that big deletion, so I’ll try again in a few hours when it’s done.
And really, that’s ridiculous. Why should it take hours to delete 10,000 messages?
Here is a recent hint from TUAW on how to rebuild the Mail database… not sure if it will help…
http://www.tuaw.com/2006/04/12/mail-slowing-you-down-rebuild-its-database/
You should be able to delete mail folders in a second by deleting the actual files. You don’t have to do it through the mail-app itself.
Add me to the list of Thunderbird pluggers.
I think that volume of messages only really gives you Eudora as a nice simple, and cheap option. It”ll handle the volume, has good filters, and is reliable and fast. It also stores messages in plain text format, so it easy to work with sans-Eudora. Eudora is also being redeveloped on the Thunderbird core as a fully open source app. It isn’t as pretty as some (well, most), but it does work. It is not good with HTML messages either.
It could also be worth looking at how much RAM your Mac has, consider 1GB to be a minimum now (some of the latest applications even need at least 4GB to be running well)
Go to
/Users/******/Library/Mail/POP-*****@pop.gmail.com/
where ***** is your name.
You can then choose which files to delete- direct from finder.
1.Burn all the archived mail onto a CD monthly.
2.Delete archived mail from MAC
I do not use MacOS but Linux, but both are more a less UN*X like. I had a similar problem with Evolution (that’s a mail client), and switched from mbox format to Maildir. This is much quicker, because in Maildir format all mails are single files instead of all mail clobbered together in one big file.
So, if there is an option to use Maildir, try that.
I suggest:
Start by a search in the mailing list archive, available from this page:
http://www.mulberrymail.com/support.shtml
Then copy your basic problem info to the support pages for Mulberry, and see what the users say.
Main page: http://www.mulberrymail.com/
http://trac.mulberrymail.com/mulberry/wiki
http://www.mulberrymail.com/faq/index.shtml
I am using Eudora for OSX, and it isn’t very beautiful but it is fast. I don’t have anything like the amount of mail you do, and occasionally clean out mailboxes, but I am on some high volume mailing lists and have never noticed any slowdown. Search speed is incredibly fast, which can be useful. The only thing I don’t like about it is the way the filters are not searchable. (But you might enjoy the chilli peppers function. I know I enjoy setting it off by using naughty words.)
My largest mailbox at the moment has almost 6000 messages in it and is no slower to open or display than the one with fifteen. I often don’t clean out my mailboxes for quite a long time, and have never noticed a slowdown.
Eudora is going open source sometime this year.
Can anyone tell me why this site hates the Safari browser? Mine crashes everytime i come here.
I hope you’ve rebuilt any painfully slow mailbox. It’s the bottom command in the Mailbox menu. Think of that as creating a new, up-to-date index so the mailbox loads much faster.
While I’d second using mutt, there’s a program called Gnumail that looks like Mail.app but apparently is faster:
http://www.collaboration-world.com/gnumail/
Part of the problem is that apparently Apple switched to an undocumented, proprietary mailbox format. You could try this script that can be used to archive and/or convert directories to mbox format:
http://homepage.mac.com/aamann/Mail_Scripts.html
I’m going to have to add to the chorus of people suggesting Mozilla Thunderbird. I use it to deal with tens of thousands of messages with no problems whatsoever. It’s by the same people who made Firefox, so it has to be good :-P
The WAAGNFNP is against merely technical solutions.
I’m not a Mac or email expert, but with mail that size, it sounds like you could use an industrial strength database to store and index your mail. One thing I’ve come across is DBMail at:
http://www.dbmail.org/index.php?page=overview
it is GPL and uses your choice of Postgresql, MySQL, or SQLite as the backend database, which are industrial grade (at least, I know the first two are).
Again, with the volume of mail, I’d recommend something that employs a good-old database backend, not a lightweight mail one (though, again, I’m generalizing, and *assuming* these are better than the proprietary email databases).
HTH
Eudora is good at moderately large mail volumes.
Quite simply, managing that volume of mail isn’t what Mail.app is designed for. It uses SQLite as an index, which is fine for most uses, but isn’t for really big volumes. If you need to be able to keep and refrence that much mail, you might consider pushing it to a mailing list archival tool – something to build you HTML indexes and whatnot. Many of those have the overhead of needing a real database as a backing store.
Mail.App slows down when I get to a certain volume of file size in my mailboxes, not file number. So I drag the folders from Mail to a Mail Backup folder and then delete in Mail. However, if you have this many, do what others have suggested and move the files in Finder, forcing Mail to rebuild.
One thing – if you store your files also on the server as I do, you’ll need to access that server using a web client or some other method to delete them there, before turning Mail back on, or it will try to rebuild the old folders.
PZ,
Maybe you could try creating sub-mailboxes with smaller numbers of messages?
Instead of a folder “foo” with lots of messages, you might have subfolders called “foo-2006”, “foo-2005”, etc.
Then just move the appropriate messages into each sub-folder. You might be able to use smart folders to create small subsets of messages that could be quickly manipulated.
Mark Pilgrim had some interesting things to say about mail and OSX. Here he explains his method for dealing with a lot of old mail. Also see how he got bitten by Mail.app, and why he eventually left the Apple fandom. Fascinating stuff, there.
PZ,
I posted a comment at bbum’s blog pointing him here and requesting assistance. I also asked if there was some way to tell Mail “I’m going to delete a zillion messages, so don’t fiddle for each message like normal; wait until I’m done.”
He works at Apple, and knows a bit about sqlite, which Mail uses.
With luck, he’ll have some advice. He’s a good egg.
“And really, that’s ridiculous. Why should it take hours to delete 10,000 messages?”
Probably because there’s a hidden Shlemiel the painter algorithm somewhere. You can bet there’s one whenever a program is taking exponentially growing time instead of linear time as it should.
http://www.joelonsoftware.com/articles/fog0000000319.html
It’s a compsci article, but the beginning should be easy to understand even without programming knowledge since he explains how it works in words. You can stop reading after the punchline in the Shlemiel joke.
As for mail program, I’d give another vote for Thunderbird. I’ve got about 50 000 mails total in mine in various folders and it copes fine. It takes couple seconds to switch to a folder with about 10k mails, but it’s nothing intolerable.
“And really, that’s ridiculous. Why should it take hours to delete 10,000 messages?”
Probably because there’s a hidden Shlemiel the painter algorithm somewhere. You can bet there’s one whenever a program is taking exponentially growing time instead of linear time as it should.
http://www.joelonsoftware.com/articles/fog0000000319.html
It’s a compsci article, but the beginning should be easy to understand even without programming knowledge since he explains how it works in words. You can stop reading after the punchline in the Shlemiel joke.
As for mail program, I’d give another vote for Thunderbird. I’ve got about 50 000 mails total in mine in various folders and it copes fine. It takes couple seconds to switch to a folder with about 10k mails, but it’s nothing intolerable.
I’ll stop sending you cartoons. That should speed things up at least a little bit.
Sorry.
Forgot – bbum’s weblog is http://www.friday.com/bbum/
his post explaining the vacuuming is http://www.friday.com/bbum/2007/03/02/vacuuming-mails-envelope-index-to-make-mail-faster/
I commented on his followup post http://www.friday.com/bbum/2007/03/04/what-about-turning-on-auto_vacuum-on-mails-envelope-index/
It sounds like a problem that would go away if you didn’t have so much mail, and I doubt there’s a good way to fix it because the fundamental problem is simply too many messages. I know you didn’t want to hear that, but there’s at least a workaround.
I had the same problem (except with only 20,000 messages, and the server is practically on the same ethernet as my computer). What I did was take all messages over three months old and download them to my hard drive, instead of continuing to leave them on the server. In Thunderbird, that means the “Local Folders;” I have no idea what the equivalent is in Mac Mail. So I didn’t actually end up deleting any of the old messages (well, they were already deleted – this was my archival Trash directory), and I could still find them if I needed to, but only on that computer’s hard drive.
So, yeah. It’s not an oversight by the software developers as much as a ridiculously overfull mailbox. Try deep-archiving the older messages locally.
I’ll also chime in for Eudora on Mac OS. I experimented with switching from Eudora to Mail or Thunderbird a couple of years ago when I upgraded to a new PowerBook, but couldn’t tolerate the performance hit — I’m impatient and have become used to keeping large mailboxes (10k-15k messages each) that I can quickly search through.
With Eudora, it’s difficult for me to detect any change in responsiveness between small mailboxes and pretty large ones. I have noticed, however, that if you leave lots of windows open, it starts to chew up RAM. It’s also not the shiniest app out there, and the filters could be a little easier to manage once you built up a lot of them. But even so, I’ve found it highly functional for quickly dealing with lots of mail.
Use DNA strands to encode several thousand emails. Then splice into Catholics. The code will exponentially reproduce (except in the rare case of a papal edict praising birth control).
To retrieve your email- take saliva swabs from the mouths of Catholic children (narrowly dodging accusations of child abuse from R. Dawkins for labeling said children). Then isolate and culture DNA. Then do whatever it is that geneticists do. Play god- I dunno.
I’ve loved Eudora for many, many years, though I’ve been tempted by Thuderbird lately. Right now, I have about 16,000 messages in my inbox, spanning > 10 years, and it works ok. I also have a large number of special mailboxes that I use frequently. The search functions are phenomenal. The Eudora filter options and mailbox system are a mainstay of my professional (not to mention personal) existence. I’m both looking forward to and dreading the upcoming change in the program.
I’ll consider sending my direct messages to you via actual (a.k.a. snail) mail in the near future. We have a relatively vast quantity of interesting postcards from around 1963 that I occasionally use when appropriate.
Indeed it is not. It’s designed for the average-to-heavy user, not the industrial strength blogger. If you’re going to continue to use Mail.app, you need to purge your mailboxes periodically and back up the messages to whatever media you want.
Accept Jesus Christ as your personal savior.
No, he won’t help you with your email situation but once you’re Born Again nobody will want to have anything to do with you. Problem solved.
Beyond that I got nothing. As usual.
As an aside, I actually use Entourage. It’s actually a decent program, and its mailing list manager is very handy when you subscribe to several mailing lists. It also now syncs with Address Book and Calendar, which is good, because iCal is pretty lame, and I like how Entourage handles its address book better.
However, I doubt this would be a good solution for you. One problem with Entourage is that the database tends to balloon to enormous sizes rather easily. (I’m guessing it has something to do with all the linking you can do between messages, contacts, events, and projects.) Mine, for instance, is over 500 MB, and I don’t have anywhere near the amount of e-mail that you do.
My suggestion would remain: periodically back up your e-mail and then clear it out of your mailboxes. I have no experience with Thunderbird, but have been tempted to try it out to see how it works. My one question would be whether it will sync its contacts with Address Book, as Entourage can. Or does it just use the Address Book database directly?
Clearly, your problem is giving priority to student mail–just delete them automatically.
In all seriousness, in my experience it’s not enough to simply trash the mailbox folder when things go seriously awry. Try backing up and deleting Library/Mail/Envelope Index (although this entails reconfiguring Mail to some extent).
Your problem, PZ, is not with the Mail application but with your limited reading ability.
You could probably get Dr. Egnor to install a second brain, sort of like a Core Duo, or maybe just the optical part (although then you’d run the risk of being called “four eyes”) I’m sure the Eggmeister would do the work pro bono for a plug on Pharyngula.
Good luck.
There is actually a site dedicated to getting the most out of Mail, and there was a note recently posted on speeding up Mail by cleaning up the “envelope index”. It’s kind of amazing to consider that there actually are e-envelopes (or perhaps e-nvelopes) associated with each email, but I saw it on the internet so it must be true.
In any event, check out their tip at:
http://www.hawkwings.net/2007/03/01/a-faster-way-to-speed-up-mailapp/
They suggest going into the terminal program and using sqlite3, the baby SQL database manager, to clean up the above mentioned envelope index. I’d do this after a complete backup, just to be safe, but if nothing else, it looks pretty harmless. It might save you from having to make more serious changes.
Apparently, so many Hawk Wings readers liked this hack that they even figured out how to automate it and hook it up with iCal.
Sometimes, just to test, I’ll make a whole new Mac user account and set up a super fresh Mail from there. This was earlier on when Mail was buggy as hell with gigantic IMAP mailboxes. It was insane and I wanted to kill myself. It healed at some point, although I did trash all my prefs and my ~/Library/Mail/ several times back when that was happening.
When you find something that works, PZ, could you let us know? I’ve got a similar problem, though not quite as extreme, with about 150,000 messages (mostly spam though, so I suppose I could get rid of that…)
I’ll umpteenth the vote for Thunderbird. On my modest powerbook (G4,1.25GHz,1.25MB) my roster of about 15k msgs over 8 or so accounts has never caused noticable slowdown or any confusion. It’s reliable over the last four years.
I used Eudora from the late 80’s to early 00’s. It finally failed both on speed and reliability issues, tho it may have since recovered. I tried mail.app for about a year, finally switched to thunderbird and haven’t looked back.
It will be a project to convert your mail.app files into thunderbird. A search in macOSXhints.com will be helpful here. Other technical discussions of interest at oreilly.com, from most recent to least:
Open-source Eudora to be derived from thunderbird
About mail.app version 3 (leopard)
Disatisfactions with mail.app
Just tried out the Hawk Wings trick on my 150,000-message Mail setup. Took just a couple minutes and there was an immediate and obvious speedup.
I run IT support for a Physical Science Division at another Midwestern research University. I hope this helps..
DON’T USE EUDORA. Quallcomm is dropping support for Eudora. They are officially killing the software this fall. They will be donating the IP for the interface to the Mozilla Thunderbird project.
The bigger problem is that Eudora does fail, not infrequently, and when it does.. it fails in absolutely inspired ways. I’ve seen Eudora erase all the mail off of an IMAP server when it crashed. Much more common though, Eudora corrupts it’s Table Of Contents file and you have to delete it to fix it. This causes all kinds of problems, like downloading dups of mail (generally not a problem with IMAP) and loosing settings like flags… DON’T USE EUDORA.
In general. Your problem is that Mail.app stores all of the mail messages as individual files. If you poke through ~/Library/Mail/ you’ll find them all as .emlx files. The HFS+ file system (and the Finder more so) are slow when dealing with insane numbers of files. Granted a typical OS X boot drive has over 100K files but they generally aren’t all in just a few directories.
Eudora would be faster because all the mail in one mailbox is in one large text file. The TOC file keeps track of each one (where it is, when it arrived, if it’s flagged..) Unfortunately, Eudora sucks.
Other common apps also use faster methods. MS Entourage keeps mail in a database (just like it’s Outlook bretheren). The problem with Entourage though is that it also sucks. We have very few Entourage users and we see a very disproportionate number of corrupted Entourage mail databases. There is a tool to rebuild the database but this always leads to mail loss.
My suggestions:
Stick with Mail.app, archive mail out of your in-box regularly, occassionally delete and rebuild your Spotlight database (Spotlight search is archiving all your mail and I’ve seen lots of Spotlight issues). This will delete and rebuild your Spotlight archive (run this in terminal.app) sudo mdutil -E /
2nd option, try thunderbird. It’s our officially recommended mail client for Windows for anyone not using Exchange Server. The current version is pretty robust. It can’t hurt to try.. aside from the hour it will take to import your mail.app mail.
Personally, me and my 12-15000 emails are happy enough with Mail.app.
Good Luck.
Steven.
I will second the notion that when Eudora fails you’re in a world of hurt in a way you won’t be when Mail.app fails (especially if you’re using Tiger)…
Equally true that when Entourage fails you’ve got a serious problem (and possibly the source of the name “E-rage you’ll hear people who more or less have to use is use.) On the other hand, the Mac BU at Microsoft has done a great job of making it feel as much like aq real Mac app as possible.
Mailsmith, if you used to manage your mail like a God in eMailer will look and feel very familiar, and not a whole lot like an OS X app. But it’s stout. (Of course, when it fails you’ll also be chiseling the mails ouyt with BBEdit or similar. You will get very used to BBEdit.)
You’ve got Pine in your terminal app, if you have X11 installed. I have my machine send notifications to pine, and I used to have the mail from my ISP delivered there… Works fine.
Or you can go to Darwin ports and install something like Sylpheed (I use this a lot, familiar if you’ve used Linux any…)
But, really, those mailboxes are too large for Mail to reconstruct its databases quickly and Mail is at least supported by Apple.
I’m in the same state as you; about 100,000 messages on a G4 MacMini. Based on my experience with running Thunderbird on my FreeBSD system at work, I’m going to move my Mac’s mail to Thunderbird as well, using my own file export utility (which requires a bit of comand-line effort to use but handles the “From” issue properly).
I like Mail.app a lot, but it just doesn’t scale enough. Thunderbird handles the 50,000 or so messages I have at work quit speedily, and has nearly all the features though not the polish of Mail.app. I can let you know how it goes if you’re interested.
“Mail struggles and strains to just display the *gorram* list of messages”
ah ha! a Firefly fan…shiny.
Thunderbird hands down, I’ve been using it on OS X for last years. It has matured to a very stable mail client, both for online and offline use.
I’ve currently 150 000 emails in local folders and Inbox is ~2000 year around. No complaints for speed, I’m running it at PowerBook 12″ G4.
I’d suggest PowerMail, which happily handles vast quantities of email (100 000+ messages in my own experience). It’s regularly updated, has excellent search and filtering functions, and a clean and efficient interface.
Here’s thought from a totally different perspective:
If your mail file management system works like a File Allocation Table (FAT), then when you “delete” a file, the information is NOT erased: all that happens is that the FAT deletes the first character in the file name and by so doing, ceases to protect the component memory from being used. As I understand it, each file consists of a series of (often discontinuous) memory sectors, and each sector ends with the address of the next one that contributes to the file. The FAT only specifies the address of the first chunk.
Even though you think you have erased the file, you have not! You still have all of the information on your disk.
Defragmenting your hard drive might help.
Did you ever try Powermail?
It can handle large mail databases, multiple accounts, mail filters…
http://www.ctmdev.com/powermail/
It is erased for all practical purposes when it comes to the mail application. The mail app. doesn’t keep track of the old deleted mails after they are deleted, and they can be overwritten by other applications.
The lack of deletion is only an issue if you want to ensure that no one else can ever read it.
Defragmentation of disks are not a bad idea in general, as it some times speed up things. As is cleaning up the registry (don’t know how necessary this is on a Mac though).
‘And the angel opened the seventh seal, and lo sinners’ mail clients shall get all ballsed up.’ End times are upon is, I prophesy.
“Use DNA strands to encode several thousand emails. Then splice into Catholics. The code will exponentially reproduce (except in the rare case of a papal edict praising birth control).
To retrieve your email- take saliva swabs from the mouths of Catholic children (narrowly dodging accusations of child abuse from R. Dawkins for labeling said children). Then isolate and culture DNA. Then do whatever it is that geneticists do. Play god- I dunno.”
And backup onto crackers!
Just use Gmail and go “after it” with your mail client (I use on my Macpro the kmail/kontact app, from the KDE “suite”, as I am an ex-Linux guy, but your native OSX mail client should work just fine), so that you can have it locally saved, as instructed here:
http://mail.google.com/support/bin/answer.py?answer=13287
If you need specific config instructions for the OSX mail client, shoot me a private email, and I will figure it out on my machine, then let you know.
You will still have to use the web interface for a little while, to be able to define (train) for Gmail what you consider spam, but then it will take care of all this for you. After a few weeks of training my profile in Gmail, I only have to log into the web interface once a month (if!), just to “triple” check the spam categorization – and I have not found almost anything to complain about. The spam on gmail gets expried/deleted automatically, so even size of your inbox there won’t be affected (rumors are that Gmail will bump up this to over 3GB, anyway)
Just a quick follow-up – did not read carefully your post – sorry about that. I now understand that your problem is not high volume of spam, but rather high volume of email that you still want to keep and manage. As I said earlier, I use kontact/kmail on my MacOSX, and I have never had a problem. I now have roughly 50,000 messages in various folders (I found maildir format being slightly better than mbox), and do not have an issue with administration … yet.
Sorry for the earlier off-topic comment.
Quick corollary..
Given the enormous number of mail files, it’s also probably not a bad idea to take your mac by your IT shop and ask them to run a scan/repair on the drive. You’ve got a lot of file turnover on that system. It certainly wouldn’t hurt to check the volume headers, file b-tree and the extents tree (which I believe is also a b-tree in structure).
It’s not terribly uncommon for the file system on HFS volumes to get corrupted. This can cause all kind of problems (unstability being common). More importantly, a good utility will optimize these data structures.
Giving it a good once-over with TechTool, DriveGenius, or a similar tool may make a big difference.
Steven.
If you’re using IMAP mailboxes, you can set up rules to move the messages to a local mailbox when they reach a certain age.
If that’s not good enough, you can archive older messages in text files, where you’ll still be able to read and search them without using Mail. They don’t make it clear how to do this–there’s no Export command on the File or Mailbox menu. What you have to do is
* select the mailbox whose contents you want to export,
* then hit Command-A or select “Select All” from the Edit menu.
* Then hit Command-Shift S or select “Save As…” from the File menu and
* choose a location and give it a name.
** You can also choose whether or not to include attachments. I suggest you use the “Save Attachments…” command and save them separately if you want to keep them.
Once you have exported everything, just hit the delete key to remove all the messages from the mailbox. This will also delete any attachments you didn’t save. You may want to rebuild the mailboxes after deleting their contents.
I don’t have to do this regularly enough for it to be a pain in the ass, but it can probably all be AppleScripted.
“Mail struggles and strains to just display the *gorram* list of messages”
ah ha! a Firefly fan…shiny.
I noticed this as well; Yet another reason to love this guy…
John
It wouldn’t help to deal with the registry, there isn’t one.
If you’re not using a tool like DriveGeniur or TechTool Pro, you should be. My first, second and third recommendation would be DiskWarrior from Alsoft.
There’s an easier way of repopulating the Mail SQL database:
1) Quit Mail
2) Go to \[Computer Name][User Name]LibraryMail
3) Drag the file “Envelope Index” to your desktop (for backup purposes only)
4) Start Mail and wait for it to recreate and populate the database
I do this once a month and Mail keeps on purring like a sweet little kitty cat… that would be a tentacled kitty-pod for you.
I second Lettuce (never thought I’d type that) in praise for Disk Warrior. It was responsible for the only resurrection I believe in.
Well that last one sounds easy enough to try.
How old is PZ’s mac? I need to do this on my Quicksilver G4 tower.
I concur with one of the earlier respondants, this sounds like a file system problem more than a mail application problem to me.
I would suggest going into single user mode (shut off your system, and hold command+s when you turn it on again) and then use the fsck -fy command to debug the filesystem. You may also want to start up in safe boot afterward to remove the files. (rough equivalent of having extensions turned off back is OS 9-) To do this, hold shift as you boot up, until you see the ‘safe boot’ message, or your login screen.
There’s no sane reason emptying your trash should take 20 min let alone two hours.
I hope this helps.
That’s odd. I’ve used Entourage for years now, and I’ve never had this problem.
Ah, the awesome statistical significance of n=1! ;-)
Thunderbird > you ;)
I shouldn’t laugh, I only have tens of thousands of saved emails in it. Over the past 6 years. I don’t delete easily…
You can try re-installing the mail program. It won’t delete the messages, and may clear some bugs.
DON’T USE EUDORA. Quallcomm is dropping support for Eudora. They are officially killing the software this fall. They will be donating the IP for the interface to the Mozilla Thunderbird project…
ooooooh!!!! Noooooo! I’ve used Eudora for more than 10 years with no problems at all. It’s light, agile and very usable. *Naturally* they are killing it. [Insert favorite expletive here]. I tried Thunderbird and hated it but that was a couple of years ago. Maybe by now they’ve fixed its many, many bugs. {sigh}
Oh, and Safari can’t handle the new ad. It blinks out the second your page is up. Firebird handles it okay, though.
Safari 2.0.4 Handles it perfectly well. Update your software.
I’ve used Thunderbird for several years, and am generally happy with it. I’ve never had 100K messages in it, but I have had over 10K and it worked fine. I also have many dozens of folders, and e-mail going back to the early 90’s totally just under a Gb (must to the U’s dismay :->).
All that said, both the people that write these programs and the U of M strongly encourage you to try to keep your IMAP (you are using IMAP, aren’t you?) inbox empty and make sure you clean up (expunge, or compact) your inbox on a regular basis. The delays you refer to really don’t make sense, and I suspect (as others have) that failure to expunge your inbox is part of the problem.
My current strategy (which I implement with varying degrees of success) is to try to keep my inbox empty (500+ is a lot closer than 10K+, right), moving things that I either want to save or just haven’t dealt with into a folder called (for this year) “2007”. You might want a more fine-grained archiving system (“2007-Mar”) given the amount you’re getting, but my understanding is that everything will run better if you can regularly empty your inbox.
Merlin Mann of 43Folders has thought about this way too much…
Rienk, you’re a lifesaver! I followed your procedure (over 375,000 messages — egads!) and although it took about 75 minutes to rebuild the Envelope Index, Mail.app now runs many times faster than it did, even in folders with 20,000 messages. Much more livable. I can forgo experiments with Thunderbird until I have an excess of free time…
I am assuming that you are using a variant of OS X? Why not use the power of Automator and/or AppleScript to automate the tasks of downloading, sorting, and archiving your e-mails?
though iam following the procedure correctly of sending messages its not going properly please help me