The Packages Build SystemFriday, October 7, 2011
A short introduction to high-level optimization
Today, most software products can be installed with one click or by simple command in the terminal, by calling a package manager. And the users don’t need to know all the hidden complexities that you can begin using a new tool without expending effort. But behind the installation systems is a complicated process and problem is not limited to simply copying a files. And more importantly, each of these files is the efforts of many people.
I joined the Intel month ago where I started working in a team dedicated to the development of the system and build of products. I saw a lot of hidden from the public. Our systems allow you to create installation packages for three popular operating systems: Apple OS X, GNU Linux and Microsoft Windows. We support all key products:
- Intel® Parallel Studio XE
- Intel® Cluster Studio
- Intel® Composer XE
- Intel® Integrated Performance Primitives (Intel® IPP)
- Intel® Math Kernel Library (Intel® MKL)
- Intel® Threading Building Blocks (Intel® TBB)
And others, full details are always available online: http://software.intel.com/en-us/articles/intel-sdp-products/
Build process optimization
Packaging software products into installation bundle is a long process. It is not difficult technically, but when you need to begin supporting the new products or enable a new libraries into existing products in short amount of time we need to act quickly. The packaging process is similar to pack files into the archive, but unlike this example, the installer must be interactive. It asks the user where install the necessary data, verifies the readiness of the operating system to install process, compares the system requirements of the product with the user machine parameters , checks the previously installed software, duplicate files and identify conflicts.
The process of packaging a large product such as Intel® Parallel Studio XE takes a long time, about two hours, depending on the hardware. The existing time limit for the packaging, which includes time for replication, testing and publication on back-end servers, said the need to have a quick packaging system.
To achieve this goal we have available a number of optimization techniques:
- Parallel packaging - allows us to simultaneously build the components of the software package, which reduces the time to 40 - 60 percent. For example, using the parallel model, packaging time of Intel ® Parallel Studio XE decreased from 2 hours 30 minutes to 1 hour 30 minutes.
- Low levels compression - is used for test builds of products and allows for winnings in 5 percent of the total build time.
- RAM disk - use RAM to store temporary files. At the moment this function is experimental. It speeds up the packaging to 5 - 10 percent, depending on the size of the files in the product.
In practice, when necessary to build the same product multiple times for testing and debugging, the build process is accelerated by disabling minor features; for example, such as verification of digital signatures - they are only needed in the final product and their presence in the test builds not necessary. It is easier to work by concentrating on the essentials.