Friday, December 16, 2011

Firefox 8 data cache cleaning

By default, my installation of Firefox uses up to 1 gb for file caching. Firefox uses a nice 16x256 (or similar) directory structure so that not all files are stored in the same directory. This is nice! However, when you ask Firefox to clear the cache, Firefox deletes files from this directory structure in (apparently) random order. This is Bad(TM):

  • That's a lot of directories! Using du -sh ? at the root cache directory will show that progress is made in a random fashion. Moreover, if you keep doing du -sh ? on the command line, then directories stay mostly cached and you can see progress is fast (meaning: slow, but not frozen molasses slow). If you wait a while, then directories don't stay cached and progress becomes (frozen molasses) slow again.
  • For some reason, it looks like OS X won't do write combining anywhere as efficiently if you delete files at random from the directory structure. The activity monitor showed about 4mb / sec disk writes throughout. However, the result was super slow deleting.

After about an hour, Firefox still had to delete about 700 mb of files. So I decided to help, and run commands such as rm `find F -type f -print` at the root of the cache directory. Apparently Firefox doesn't mind not finding files it wants to delete, and rm keeps going if it can't find a file to delete, so things went well. The rm command deletes files sequentially in the same directories, so I expected it to run faster. And yes indeed, it was quicker by a lot. I was done deleting the rest of the files in no time. I should also point out that, even though at times there were effectively two rm threads (Firefox, and the command line), actual disk i/o throughput was not higher and remained at about 4 mb / sec. Nevertheless, actual deleting performance was way faster.

Firefox developers: please delete cache files sequentially, yes? Also, Memshrink is awesome and it makes a huge difference, please keep going!

No comments: