Tuesday, September 2, 2008

ccache = much speedier Firefox builds

I just discovered the awesome tool "ccache". I wish I'd known about it long ago -- it drastically speeds up Firefox rebuilds. From its man page, ccache "speeds up re-compilation of C/C++ code by caching previous compiles and detecting when the same compile is being done again."

I figured I'd share my ccache setup under Ubuntu Linux, in case anyone else wants to use it. (This is based on Darin Fisher's setup, from his ccache post back in 2004.)

Install ccache:
sudo apt-get install ccache

Add these to ~/.bashrc:
export CCACHE_DIR=/scratch/work/builds/.ccache
export CCACHE_HARDLINK=1
export CC="ccache gcc"
export CXX="ccache g++"

(Note that "/scratch/work/builds/.ccache" is just an empty directory on the partition where I intend to build Firefox. You'll probably want to replace that with something else -- or you can remove that line to get the default location, $HOME/.ccache)

Then, source that file (to make sure $CCACHE_DIR is defined) and initialize your cache with a largish maximum size (2 gigs):
source ~/.bashrc
ccache -M 2G
And you're done! The next time you do a "make -f client.mk build", you should see "ccache g++" in place of "g++" in the output, and your .ccache folder should start to grow in size.

I tested ccache's performance by doing an initial build of Firefox trunk (to populate the cache), deleting that build, and then doing a second full build (to make use of the cache). I timed the second build, to see how much the caching helped.

On my work machine, ccache reduced build time from 22 to 11 minutes. And on my home machine, ccache reduced build time from 12 to 4 minutes! (Compare that to multi-hour build times on Windows. :) )

9 comments:

Gary Kwong said...

I don't think anything else other than |sudo apt-get install ccache| is needed for 8.04.1 Hardy anymore. I just tried with all the other options from the 2004 post as listed above and it gives me the error "configure: error: installation or configuration problem: C compiler cannot create executables".

I deleted them all, rebooted the machine, then ran make without all those options. It then works perfectly.

See http://zenit.senecac.on.ca/wiki/index.php/Improving_Build_Times

Gary Kwong said...

Oh second thoughts, I think it was my error. I had (epic failure) failed to change to my .ccache location, instead copying from your /scratch/work/..... path.

Epic failure!

(your ccache instructions work brilliantly now, my previous comments above will make it compile _without_ ccache support).

:)

Daniel said...

Hi Gary -- cool, I'm glad you got it working! :)

Timothy said...

Thanks for making this post, was helpful for me.

For some reason export CC=... didn't work for me.

What did work was adding /usr/lib/ccache to the start of my path by adding the following line to my ~/.bashrc

export PATH="/usr/lib/ccache:$PATH"

/usr/lib/ccache is created by ccache and just contains symlinks to ccache for a bunch of compiler names (gcc, g++, etc).

Daniel said...

Timothy -- that's strange that "export CC=..." isn't working for you. Perhaps your mozconfig file sets CC / CXX to something else? (which would override the environmental variable set in .bashrc)

The /usr/lib/ccache tip is handy, though -- thanks for that!

Kurt Schultz (supernova_00) said...

Can you do this for official builds?! :D

Sid said...

ccache is definitely very cool.

BTW, ask dolske about how he got a build from scratch in 11 minutes on Windows. ;)

Mook said...

Hmm, on Fedora (I think?), just installing the ccache pacakge will set up symlinks early in the $PATH so that gcc &c all actually runs ccache, which will then run the real compiler. I find that easier than setting up the environment variables, but YMMV.

Anonymous said...

@Mook,


cp ccache /usr/local/bin/
ln -s ccache /usr/local/bin/gcc
ln -s ccache /usr/local/bin/g++
ln -s ccache /usr/local/bin/cc
ln -s ccache /usr/local/bin/c++


Is indeed a convinient way of doing things if you want to use ccache all the time. But remember that the strenght of ccache only show on rebuilds, not for one time compilations where it actually adds a little overhead. So on a source distribution you may end up losing time.

Regards,
Mirrakor