autotools

Autotools

Details: Last Updated: Sunday, 10 March 2019 15:56; Published: Monday, 18 February 2019 00:12; Hits: 894

Autotools:

We saw the make utility for building (compiling and generating executable) large programs. However, usually we write Makefile for compiiing programs for the system on which we are running make. i.e if I am working Linux OS, executable I generate is for Linux OS. However, that same executable will not work on Windows, and even on other flavors of Linux OS. This is because same compiler version, lib files, etc may not exist on Linux systems. So, we need different Makefile tailored for each platform. This will create too many Makefile each unique for each system.

To remedy this situation, we need to have a Makefile which will have #ifdef for different platform, and will generate executable differently for different platform (i.e if older version of gcc exists on some system, it may not support some flag. In that case, ifdef will help resolve such issues, and still let the Makefile build an executable). Having all these ifdef in Makefile and making sure that such pgm work on all arch is very tedious task. So, most of the software that is intended to be distributed for multiple platforms, use a tool called "Autotools". This tool automates the task of generating binary for different platforms.

Most of the Linux programs that we download, have their binary executable or source code. The programs which have their binary executable directly available, we do not need to do anything. We just run the executable and pgm starts running. However, programs which have their source code available, we usually need to run 3 steps as a end user to generate executable. This is called "GNU Build". Installer on your system unpacks the downloaded package (if tar.gz file, then use gunzip and tar, if .deb pkg, then use dpkg or apt, etc) and then runs these 3 steps:

./configure => analyzes your system to see what kind of programs and libraries you have, so it knows how to build the program best
make => actual building is done using Makefile generated from above step (same way as we do using make with a Makefile)
sudo make install => installs the pgm (puts pgm libs,binary,etc in appr dir with appr permissions). By default, it's put in /usr/local/ (bin in /usr/local/bin, lib in /usr/local/lib, etc)

These 3 steps are needed because over here Makefile gets generated differently for different platforms. ./configure generates a Makefile then make runs on this Makefile to generate executable, and then "install" puts generated executable in appropriate dir. We may never need to write a program that we distribute to other people, so you may wonder why learn Autotools. Reason is that most of the times we end up using these pgms, which requires us to run these steps. Having a brief understanding of this tool "Autotools" helps us when building (compiling and installing) 3rd party pgms on our system. Learning Autotools is a full time job in itself, so I'll just highlight few basic cmds with an example.

Full detailed tutorial for autotools here: https://www.lrde.epita.fr/~adl/autotools.html

Brief tutorial on this is: http://markuskimius.wikidot.com/programming:tut:autotools

Autotools is a collection of three tools:

Autoconf — This is used to generate the “configure” shell script. As I mentioned earlier, this is the script that analyzes your system at compile-time. For example, does your system use “cc” or “gcc” as the C compiler? Full Autoconf doc here: https://www.gnu.org/software/autoconf/manual/autoconf.html
Automake — This is used to generate Makefiles. It uses information provided by Autoconf. For example, if your system has “gcc”, it will use “gcc” in the Makefile. Or, if it finds “cc” instead, will use “cc” in the Makefile. Full automake docs here: https://www.gnu.org/software/automake/manual/automake.html
Libtool — This is used to create shared libraries, platform-independently. No need to know this as it's complicated topic for advanced users.

Autotools build process has some standard things in GNU build. Good to know these:

A. make options:

1. Std Makefile targets: make all, make install, make uninstall, make clean, make check=> To make targets when pkgs have been downloaded to your system

2. For making distribution: make dist (creates a tarball named *.tar.gz by collecting all src/other files, which is ready for distribution), make distcheck (to check the pkg for any errors/issues), make distclean.

3. staged installation. using DESTDIR, we can divert install step of "make install" to other dir than the ususal dir. Then we can choose and move files to whichever dir we want.

ex: make DESTDIR=~/scratch install

B. configure options: configure --help gives all options, few important ones are listed below.

1. Std Directory var: var=prefix (default is /usr/local). By chnaging value of this, we can put bin, lib, doc, etc in other dir.

ex: ./configure --prefix ~/user => puts bin "hello" in ~/usr/bin,/hello, etc.

2. Std configuration var: CC, CFLAGS, LDFLAGS, CPPFLAGS.

ex: ./configure CC=gcc3 .. => configure automatically chooses appr default values for these, but sometimes we may want to override defaults.

3. Parallel build tree: GNU build system has 2 trees: source tree and build tree. Source tree is the dir containing "configure" which has all src files. Build tree is dir where "./configure" is run creating object files and other intermediate files. Most of the times, we run "./configure" in same dir where configure is located, so source and build tree are same. But, if we want to keep our source files uncluttered from generated files, we can have build tree in separate by doing this:

ex: ~/.../top-dir-pkg (this is dir where you extracted files, and has configure script). "mkdir build", "cd build", run "../configure" and "make" in build dir. This keeps all generated files in build dir, keeping main source dir intact.

4. cross compilation: To generate binary for a different system that one where we are compiling the files. By default, binaries are generated for the same system, where we compile the files.

ex: ./configure --build=i68cpc --host=solaris => Here, build denotes our sytem, whereas host is the system for which we generate binaries. For binaries to get generated for host system, cross compiler has to exist on native system, else it will error out.

5. pgms can be renamed by using --program-prefix, --program-suffix (i.e instead of installing a pgm with name "tar", we can install it as "my-tartest, by using prefix=my-, suffix=test, to prevent overwriting "tar" that is already installed)

Simple example of building a pkg: This ex shows how to build pkg like *.tar.gz from source files that can be distributed.

Autotools is installed by default. We can check version number of Autoconf/Automake by running autoconf/automake.

$ autoconf --version
autoconf (GNU Autoconf) 2.69

$ automake --version
automake (GNU automake) 1.13.4

$ autoreconf --version
autoreconf (GNU Autoconf) 2.69

ex1: write a C pgm (hello.c) that has gettimeofday function. We need to run autotools in same dir as pgm hello.c. These are the steps for running autotools. The whole goal of autotools is to generate 2 files: configure and Makefile.

0. write C pgm as below called hello.c:

#include <stdio.h>
#include <sys/time.h> // this is added on purpose, so that we can make this system dependent

int main(int argc, char* argv[])
{
   double sec;
   struct timeval tv;

   gettimeofday(&tv, NULL); // This function only exists in sys/time.h, so if that file doesn't exist, this will error out
   sec = tv.tv_sec;
   sec += tv.tv_usec / 1000000.0;

   printf("%f\n", sec);

   return 0;
}

1. autoconf + automake steps =>

Autoconf has a series of steps to generate configure script. configure is a very large bash script. Autoconf needs configure.ac as an i/p file to generate configure script.
we can write configure.ac as below: AC_* and AM_* are M4 macros. AC_* are autoconf macros, while AM_* are automake macros.

AC_PREREQ([2.69]) => Autoconf version number (optional)
AC_INIT([hello-pkg], [1.0], [This email address is being protected from spambots. You need JavaScript enabled to view it.]) => pkg name, version, email addr for bug reporting
AC_CONFIG_SRCDIR([hello.c])
AM_INIT_AUTOMAKE([-Wall -Werror foreign]) => NOTE: this is automake macro (AM_*), not autoconf macro (AC_*). options inside are optional. We turn on all Warnings and report them as error by using -Wall and -Werror. -foreign allows us to proceed even w/o having files as README, AUTHORS, NEWS, etc. Else, automake will complain about these missing files and won't allow us to generate pkg. Also, if autoconf is run with this AM_* macro, it will error out as "undefined macro"
AC_CONFIG_HEADERS([config.h]) => causes the configure script to create a config.h file gathering ‘#define’s defined by other macros in configure.ac. This config.h file can be included in hello.c file and then those defined strings in our program to make it portable.
AC_PROG_CC => causes the configure script to search for a C compiler and define the variable CC with its name
AC_CHECK_HEADERS([sys/time.h]) => checks for header files
AC_CHECK_FUNCS([gettimeofday]) => checks for library func in src files
AC_CONFIG_FILES([Makefile]) => list of all Makefiles that should be generated from Makefile.in file. If Makefile are in nested dir, provide all those here
AC_OUTPUT => closing command that actually produces the part of the script in charge of creating the files registered with AC_CONFIG_HEADERS and AC_CONFIG_FILES (i.e config.h and Makefile)
Aytomake generates Makefile.in. It needs Makefile.am and configure.ac as i/p to generate Makefile.in.
Makefile.am can be simple file specifying o/p binary file, and i/p C pgm as shown below.

bin_PROGRAMS=hello
hello_SOURCES=hello.c

2. autoreconf --install => With these 3 files above (hello.c, configure.ac, Makefile.am), we can now run autoconf on configure.ac to generate configure. Then we can run automake on Makefile.am and configure.ac to generate config.h.in and Makefile.in. But it will require lot of work to get it working. Autoreconf is a script that calls autoconf, automake, and a bunch of other commands in the right order. So, this is the preferred step instead of running autoconf and automake separately. This step creates configure, config.h.in, Makefile.in files. It also bunch of other files as install-sh, depcomp, missing, aclocal.m4 and dir autom4te.cache.

3. configure => At his point build is complete. Steps 3 and 4 are what the user would run on any system that the package is downloaded to create executable. We run these steps here to check that everything runs OK. With 3 files (configure, config.h.in and Makefile.in) generated in step 2, running ./configure script (generated in step 2), creates Makefile (from Makefile.in) and config.h (from config.h.in). These 2 files have been created after probing the system, so these files are runnable on this system. There are also extra files created called config.status and config.log

4. make => running make generates executable hello (shows the actual steps). hello.c and hello will be the files generated by this step.

5. make install => We do not run this step as it will install binaries in appr dir, which we do not want on our system. Stpes 3,4,5 are run by folks downloading our pkg.

6. make distcheck => ceates final *.tar.gz distribution pkg as "hello-pkg-1.0.tar.gz"

Now that we have the final pkg, it can be dsitributed to anyone. However, our program is not yet portable for all systems, as there are function in our hello.c pgm that may not be present on some systems in the C library. config.h is the file that comes to our rescue here. It looks at all functions in pgm, and provides us with constants in form of "#define" that we can use to check if the system has that function in C library or not. For ex: looking in config.h, we see these lines:

/* Define to 1 if you have the `gettimeofday' function. */
#define HAVE_GETTIMEOFDAY 1

/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1

Now, in our C pgm, we can use these constants to check for the existence of these on that system. So, we modify our C pgm to make it portable.

Our modified C pgm looks like this:

#include <stdio.h>
#ifdef HAVE_SYS_TIME_H
#include <sys/time.h>
#else
#include <time.h>
#endif

int main(int argc, char* argv[])
{
   double sec;
#ifdef HAVE_GETTIMEOFDAY
   struct timeval tv;

   gettimeofday(&tv, NULL);
   sec = tv.tv_sec;
   sec += tv.tv_usec / 1000000.0;
#else
   sec = time(NULL);
#endif
   printf("%f\n", sec);

   return 0;
}

Now, since we modified our pgm, we need to rerun step 3 and 4 to make sure our pgm still compiles fine. Then we can run step 6 to create tar.gz that can be distributed.

----------

OPTIONAL: The steps below are alternate set of steps that are not recommended. But they are good to know, incase we do not want to run autoreconf, but instead plan to run autoconf and automake separately.

A. autoscan => generates configure.scan. It's a small file. It should look very similar to configure.ac file above. It has all autoconf macros only(i.e AC_*). We will need to add automake macros to it (AM_*), Rename it as configure.ac to use it in flow above.

B. autoconf => uses configure.ac to generate configure. If needs config.h.in and Makefile.in . If we do not want to write config.h.in from scratch, we can use autoheader to generate config.h.in. Makefile.in contains very basic Makefile isntructions, which are used to generate Makefile.

C. autoheader => generates config.h.in. It just has few constants which are undefined.

D. automake => generates Makefile.in from Makefile.am.

E. aclocal => There will be lot of errors in automake step above. 1st set of errors will be Automake macros which aren't found in configure.ac. If we add these macros in configure.ac, then autoconf will freak out, since it doesn't know these macros. To remedy this, we provide defn of these macros in aclocal.m4. We run aclocal to creeate aclocal.m4 automatically with defn of all these macros in automake.

At his point, we have config.h.in and Makefile.in. So, configure script can run now, followed by make.

G. run configure: . ./configure => generates config.h from config.h.in, and Makefile from Makefile.in. config.h will look same as config.h.in, except that all constants are #define now. Makefile will look same as Makefile.in.

H. run make => Once Makefile is generated, we can run make. It uses Makefile to run 1st target "hello". make all => generates executable hello using rules in Makefile. Now we can run ./hello to get executable running.

Nav view search

Navigation

Search

Autotools