475GB of Email!

ByMike Moffit

A few weeks ago a customer checked in an iMac with symptoms of increasingly slow response and occasional kernel panics. The initial evaluation turned up the probable cause pretty quickly—the hard drive was completely full. This can affect performance in a couple of significant ways. First off, computers use free hard drive space for what is called virtual memory—essentially using the drive as extra RAM, and lots of it. When your drive gets full or close to it, that drive space cannot be used for virtual memory anymore and performance suffers significantly. Another effect of a full drive can be software and directory corruption due to the machine not being able to write changes to disk when the machine is shut down. These errors accumulate and can lead to apps not opening, lost preferences and a host of other issues including kernel panics, which this machine was experiencing.

Anyway, back to the story at hand: The usual suspects when the drive is full are pictures, music and movies. Those are the things everyone has the most of, and they take up lots of space. This customer had almost nothing on the machine, however. I changed the Finder view options to “calculate all sizes” and took a quick trip through the hard drive. With this option selected and using List view, you can find big folders pretty quickly. I saw that that his mail folder was 475GB, which was obviously why his drive was full. There was a folder inside his Mailboxes folder called Recovered Messages that had hundreds of copies of the same few messages. I tried to open one, but they were all corrupted. I deleted the messages, which freed up his drive, reinstalled his OS to clear any OS-related directory issues and observed for 24 hours to see if the problem recurred. It did not, so I figured it had to do with the message being corrupted and simply deleting and reinstalling had resolved the issue. Home went that machine, and on I went to the next repair.

A week later another machine came in with the same issue and had the same symptom of a large Recovered Messages folder filling up the drive. One machine I would consider a fluke but two raises suspicions, so I started doing some research and eventually found not only other people with the problem but fortunately a resolution as well. It turns out there is a bug in Mail somewhere that very occasionally causes Mail to keep recovering a specific message over and over until the drive is full. It seems to be only related to IMAP servers and triggered by large messages. The definition of “large” appears to mean “big enough to exceed your mail server’s message size limit,” whatever that may be. The message gets bounced back and the error causes it to be replicated over and over into the Recovered Messages folder.

The fix turns out to be very simple but involves a little command-line work in Terminal. First though, turn off your AirPort or unplug the ethernet cable—the machine must be offline while you do this or the problem will come right back. The offending message actually lives in an invisible folder and deleting it is the key to resolving this issue. So the next thing to do is to get your Mac to show you all the normally hidden files and folders.

In terminal type:

defaults write com.apple.finder AppleShowAllFiles TRUE

and hit enter.

Then type:

killall Finder

and hit enter. Now all the folders normally hidden will show up.

The next step is to navigate to:

~/Library/Mail/IMAP-username@yourmail.tld@imap.whatever.tld/.Offlinecache

Inside the offline cache folder you will find the offending message. Delete it and any other messages in there.

Hide all your hidden files again:

defaults write com.apple.finder AppleShowAllFiles FALSE

Hit enter, type:

killall Finder

and hit enter again. Now everything is back to normal. Delete ~/Library/Mail/Mailboxes/Recovered Messages and take your machine back online. All should be well.

After resolving this for the second customer, I contacted the first customer who had been slowly losing space again and got him up and running in just a few minutes.

Similar Posts

  • Happy Tuesday,

    This is my first time writing to all of you readers. I want to introduce myself as the new Services Manager here at Small Dog. I also want to share my awesome Christmas card picture of me and my cat with you. I know that I should have a dog, but who can resist something so soft and cuddly? Especially in that festive hat!

    Ok, so the picture is not mine. I was photoshopped into it by the marketing department because I did not wear a “*Halloween costume*”:http://blog.smalldog.com/article/2011-halloween-costume-contest/. Unfortunately, I did not win any prizes for the humiliation that I suffered at their hands. But I did get a great laugh, and I hope you do as well.

    I did make it out to the sock sale in Northfield. I was able to snag five pairs of socks for under $10. I pushed and shoved my way through the little old lades to get to the $1 pile. I then spent twenty minutes sifting through the pile, finding the best of the damaged socks. I wore those socks all day Sunday.

    Sunday was probably the last day to go ride my motorcycle this year, so even though it was raining, I braved it to go for a ride with Small Dog’s CEO Don and his wife Grace. We had a nice ride of about 80 miles through Stowe, Morristown, Elmore and Worcester. It does get dark up here fairly early, so by 4 p.m. it was starting to get cold. But with my new socks, my toes were nice and cozy.

    Thanks for reading Tech Tails. We all hope you appreciate the articles we are writing to you.

    Jason
    “*jasonl@smalldog.com*”:mailto:jasonl@smalldog.com

  • The Great Debate: Mac vs. Windows vs. Linux

    I thought I would take a minute to write about the whole Mac vs. Windows vs. Linux debate.

    Everyone has their own personal preference, but essentially I see the divide between operating systems like the divide between different flavors of ice cream. At this point, most of the mainstream OSs, (Windows, OS X, Ubuntu, etc.) all have the same basic functionality that most users need. The things that make them different __(on the user side)__ are compatibility with software/hardware and the differences in user interface layout. Having used both Mac OS, Windows 7 and Ubuntu Linux all for quite some time now, I’ve found that most of the time I’m really just using Chrome and maybe iTunes on a day-to-day basis and only having to deal with more complicated software on occasion.

    For a good everyday use OS, I recommend Ubuntu Linux to users who are moderately tech-savvy and want more customization than Mac OS or Windows provide. Ubuntu is also completely free and, of course, fully open source. The issues I have run into with Linux are mainly in software compatibility. For example, Microsoft Silverlight (the framework that makes Netflix streaming possible) is not supported on any systems other than Mac OS and Windows (XP, Vista, 7). I also am unable to run the newest version of iTunes on Ubuntu (WINE, the windows emulation software for Linux, only supports up to version 8 of iTunes at this time. If you have an older iPod and want to sync it with a Linux machine, RythmBox is a great alternative program that will let you manage your media library. If you have an iPhone, or even an iPad for that matter, you’re just out of luck.)

    Linux is great. It’s extremely customizable (down to every last part of the user interface and installed software). It gives you great core functionality, wonderful performance and ease of use. However, the limitations of software available make Linux difficult to use as a main operating system.

    Windows, as we all know, is very clunky. It’s based on a 20-year-old system that has huge security risks and fundamental issues in this day and age. Imagine everyone using touch-screen abacuses in 20 years, and someone using a solar-powered calculator from 1990. It was great for the time, but the software needs to be really stripped down and rewritten from the ground up for Windows to improve at all.

    Mac OS X is very tempting for a lot of users. It has the compatibility of Windows (Silverlight, iTunes, Photoshop, etc.) and the security of a Linux OS (mainly stemming from Apple’s constant security updates and the fact that OS X has a smaller market share). Unfortunately Mac OS doesn’t really give you much customization (although it’s pretty much made for everyone), and there are still many pieces of software only available for Windows; however I find that if you are willing to pay a little more and get a Mac, you will appreciate the ease of use and simplicity of the system.

  • Follow Up: Lion Recovery

    In my last Tech Tails article I gave instructions on how to create a bootable Lion installer on a flash drive. I received a lot of follow-up emails from people who have purchased a new Mac that came with Lion and who don’t have the Lion installer to work with. For these folks, their machines don’t even come with restore discs; everything is done through the hidden Lion Recovery partition that’s on that drive. This is great for most things–you’d just hold down the Option key when booting and choose the Recovery partition if you needed access to the utilities or needed to reinstall your OS. The problem with having this hidden partition is that you lose it if your hard drive fails.

    Apple does have a tool that hasn’t been widely publicized that will help you get around this. They have a Recovery Disk Assistant application that will help you create the Recovery partition on an external drive. The process will erase the external drive and create that hidden partition on that drive, and it will still be on your internal drive as well. Some things to consider in the process: you’ll lose around 1GB of storage on that external drive, and if your computer came with Lion, you’ll only be able to use the external Recovery drive with your computer. However, if you’ve upgraded your Mac to Lion, you can use your Recovery drive with any system that was upgraded to Lion. I do not believe you can use this to upgrade a machine, however, so you’ll still have to purchase Lion if you don’t already have it.

    You can download the Recovery Disk Assistant from Apple’s Support site “*by clicking here*”:http://support.apple.com/kb/HT4848.

  • iTunes Match: First Impressions

    iTunes Match is a music syncing service similar to Spotify. For twenty-five bucks a year, you can get access to your entire music library from any of your iOS devices anywhere in the world, as long as you have network access. When you set up iTunes Match, iTunes will compare your library with Apple’s iTunes Store and match any song that you have that appears in their catalog. Any tracks that Apple doesn’t have will be uploaded to the iCloud servers for you to access. For most people, there won’t be much that you’ll need to upload unless your collection is filled with very obscure artists. Apple’s iTunes Store has over 20 million songs, so chances are your music is in there. I have just under 23,000 tracks in my library, and around 6,000 of those tracks were not in the iTunes Store. It took a long time to sync my library: it took nearly three days for it to finally match and upload everything. Since then I’ve been able to access every track I have on both my iPhone and my iPad, no matter my location; my phone would even get the songs from a 3G network (luckily I’m still on an unlimited data plan). The sound quality is good, however it can take a couple of seconds for each song to begin as it needs to buffer. You do have the option to download songs or entire albums to your device from the cloud. I haven’t had to do this yet, so I can’t comment on it.

    Depending on your collection, your devices and whether or not you want to drop $25 a year on the service, it may or may not be useful to you. For me, I have a huge music collection that won’t all fit on my devices, so this is pretty useful. I’m sure there will be bugs with the system that haven’t shown up yet, but knowing Apple they’ll be fixed relatively quickly. I think that this service is going to be pretty cool, and for $25 it’s a steal, especially when they’re now storing 20-30GB of my songs for that price. While it’s not a backup of your music, it’s nice having easy access to the entire library.