Thursday, October 8, 2009

Lots of optimisations

While mass-building for F-11, we realised that there are a lot of areas we can work on to make the build process faster. ARM boards are not as powerful as the i686 machines, so, the actual build time cannot be reduced beyond a certain limit. But, we can increase the throughput. So here's what we've been working on for some days.

1) We got some new builders in the build-pool. Each one has 2 GB RAM and 1200 bogoMIPS processor. So, now we have 8 builders total in the build pool. We hope to add 2 more in some time. With 10 builders, the mass-builds would be much much faster.

2) Eliminate buildSRPMfromSCM tasks. Frankly, they were more of a red-tape. buildSRPMfromSCM is already done on the fedoraproject koji, it would be much much better (and faster) to just pick-up the SRPM and build. So, we wrote wrapper scripts to do just that. This has been of great great help because the boards are always doing some "meaningful" work now.

3) Optimise postgresql database. We vacuumed (fully) the database and reindexed it and it is blazing fast.

4) Also, we have scripts to partially automate analyzing build failures, determining which library sonames have changed upstream so that we need to build them first, etc. The build process is now fairly defined and streamlined.

And we have already started working on the base-set of F-12!