Aside from Gnu Autotools, one other tool that is very popular for building projects is "CMake". Just like GNU Autotools, it's open source and cross platform. It also provides support for testsing and packaging using Ctest and Cpack. Cmake project started in 2000. The executable pgms Cmake,  Cpack and Ctest are all written in C++. CMake/CPack/Ctest are case insensitive, meaning that we can write name of pgms, files or text in files in any case, and it will mean the same (for ex: CMake can be written in any case, i.e cmake, Cmake, etc, and it still runs correctly).

Links:

https://cmake.org/cmake-tutorial/

https://www.johnlamp.net/cmake-tutorial.html  => whole tutorial also avilable on pdf: https://www.johnlamp.net/files/CMakeTutorial.pdf

 https://ecrafter.wordpress.com/category/programming/

https://ilcsoft.desy.de/portal/e279/e346/infoboxContent560/CMake_Tutorial.pdf

Just as Make uses Makefile which is used to build source code, CMake has CmakeLists.txt files. These describe your project to CMake. Cmake generates Makefiles for you in Linux, instead of we writing it manually. Then we run make using these Makefiles to create executables and libraries, just as we do with regular make. These Makefiles are native build files (i.e generated based on what's available on current system), so can be run on that system (hence called cross platform)

CmakeLists.txt are fairly simple especially compared to Makefiles.Note Cmake is also inteneded for other platforms as Microsoft Windows and Apple Mac, so it is used to generate other build system other than Makefiles. We'll talk only about Unix Makefile generation using Cmake, as that is what Cmake is used for on Linux platforms.

A simple 3 line CmakeLists.txt is as follows:

CMakeLists.txt: NOTE: End of each line is signaled with newline, no ; or anything else needed to signify end of line

cmake_minimum_required(VERSION 2.8 FATAL_ERROR) #This line is optional, but is strongly recommended. This adds version number. 2.8 as version number. Last option "FATAL_ERROR" is optional. This whole line says that if version of cmake is < 2.8, then cmake would error out with FATAL ERROR. If you aren’t sure what version to set use the version of CMake you have installed (cmake -version). It is recommended that this command be used in all top level CMakeLists.txt

#This is comment. Comments can also be used at end of cmd on same line as shown in above line. Multi line comments are as #[ ... #]
project("My tutorial") => Name of project, if name has spaces, put them inside " "

add_executable(tutorial tutorial.cxx) => This command tells CMake you want to make an executable and adds it as a target. The first argument is the name of the executable and the rest are the source files. You may notice that header files aren’t listed. CMake handles dependencies automatically so headers don’t need to be listed. 

 Once we have this file in a dir, we can run cmake. We will do an out of source build, where we create all new files in a separate dir. This is a recommended approach for any build software, as it keeps generated files separate from source files. We can also do in source build where we create all new files in same dir, but in source build are not preferred.

Let's start with same example as of the tutorial in Make:

 /home/aseth/ => cp hello.c and config.h in this dir.

Create CMakeLists.txt as below, with only 2 lines:

project("My tutorial") # Name of project,if multiple words, separated by spaces, put them inside " "
add_executable(hello hello.c) #name of config.h is not put here, as CMake handles headers automatically

mkdir build
cd build/
cmake -G "Unix Makefiles" .. => This creates "Makefile" for us in same dir as "Makefile". (option -G provides Generator name, here we create Unix Makefiles as generator). It also creates couple of subdir here. ".." is needed since that specifies path of CMakeLists.txt, which is 1 level up. Here we are doing an out of source build as we are in a separate dir build. If we were doing an in source build, by running this cmd in main dir (/home/aseth), we would have to replace ".." with "." as that specifies the path of CMakeLists.txt which is now in current dir for in source build.

Build package => similar to AutoTools
make VERBOSE=1 => runs make as you would with a Makefile created manually. Makefile created above by Cmake is quite fancy. It suppresses the standard output. While this provides a neater and cleaner experience it can make debugging more difficult as you can’t check the flags passed to the compiler, etc. We can get all of that output by running make VERBOSE=1.

NOTE: in make cmd above, we didn't specify a target. If you look at Makefile generated by cmake above, you will see it has lot of std targets as all, clean, etc. By default, target "all" is run, so above cmd is equiv to "make -all VERBOSE=1"

executable hello is created in this build dir. object files for hello and other related files are all created in CMakeFiles/hello.dir subdir.

Install package => similar to AutoTools

make install => we can omit build step above, as both build and install can be handled by this cmd

CMakeCache.txt => this is one other important file created during build process, in build dir itself. It has entries of form: VAR:TYPE=VALUE. It speeds up build process. There is no need to modify it manually.

Syntax: All files in CMake are written in CMake syntax. All details of syntax can be found on CMake website. We'll just discuss important ones.

Variables: Cmake comes with list of predefined var as CMAKE_BUILD_TYPE, etc. These Variables can be changed directly in the build files (CmakeLists.txt) or through the command line by prefixing a variable's name with -D (i.e -DBUILD_SHARED_LIBS=OFF )

CMAKE_MODULE_PATH => Path (dir) to where the CMake modules are locate. These CMake modules are loaded by the the include() or find_package() commands before checking the default modules that come with CMake. By default it is empty, it is intended to be set by the project.

CMAKE_INSTALL_PREFIX => Where to put files when calling 'make install'

CMAKE_BUILD_TYPE => Type of build (Debug, Release, ...)

BUILD_SHARED_LIBS => Switch between shared and static libraries

We can change predefined var or define our own variables using set cmd. Data type for variables are strings and list (which are lists of strings). All var can be accessed via ${VAR}.

set(STRING_VARIABLE "value")

set(LIST_VARIABLE "value1 value2")


To print the value of a variable, use the message command and deference the variable using ${}:

message("The value of STRING_VARIABLE is ${STRING_VARIABLE}")

ex:

set(CMAKE_CXX_FLAGS "-Wall -std=c++0x") => this sets additional compiler flags to be applied to both compiler and linker. This var is predefined var, but we can assign any value to it.

Conditional constructs:
1. IF() ... ELSE()/ELSEIF() ... ENDIF() => ex: IF (UNIX)

2. WHILE() ... ENDWHILE()

3. FOREACH() ... ENDFOREACH()

CMake Modules: Special cmake file written for the purpose of finding a certain piece of software and to set it's libraries, include files and definitions into appropriate variables so that they can be used in the build process of another project. (e.g. FindJava.cmake, FindGSL.cmake)

 These modules are in dir where CMake is installed. On my system, cmake is installed in /usr/local/ dir, just like anyother pkg installed.
dir: /usr/local/share/cmake-3.14/Modules/ => there are hundreds of modules here as *.cmake and *.in
 
Ex: FindGSL.cmake ( /usr/local/share/cmake-3.14/Modules/ FindGSL.cmake ) => Finds native GSL includes and libraries, and sets appr var. The file reads like this:
 
include ( ...file.cmake) => includes a cmake file to handle args passed to the module
if (GSL_ROOT_DIR defined, use it),
else use PkgConfig module => find_package(PkgConfig), pkg_check_modules( GSL QUIET gsl ))
set GSL_INCLUDE_DIR, GSL_LIBRARY, GSL_CBLAS_LIBRARY, GSL_INCLUDE_DIRS, GSL_LIBRARIES, (debug versions too)
If we didn't use PkgConfig, try to find the version via gsl-config or by reading gsl_version.h. set GSL_VERSION
Now, handle the QUIETLY and REQUIRED arguments and set GSL_FOUND to TRUE if all listed variables (GSL_INCLUDE_DIR, GSL_LIBRARY, GSL_CBLAS_LIBRARY, GSL_VERSION ) are TRUE

On running, FindGSL module will set the following variables in your project (other modules may set other similar var, i.e OpenCV will set similar var, but starting with prefix OpenCV, i.e: OpenCV_FOUND, etc):

GSL_FOUND          - True if GSL found on the local system
GSL_INCLUDE_DIRS   - Location of GSL header files. => all include dir. Here, it's same as GSL_INCLUDE_DIR
GSL_LIBRARIES      - The GSL libraries. => all library, here it's both of these: GSL_LIBRARY, GSL_CBLAS_LIBRARY
GSL_VERSION        - The version of the discovered GSL install.

Packages: Packages provide dependency information to CMake based buildsystems. Let's say we installed a package for OpenCV. This package might have been built with CMake or may been built with any other pkg building software as AutoTools, etc. If package was built with CMake, then , then it has "<name>Config.cmake" (i.e OpenCVConfig.cmake) file associated  with it. This is called as Config-file package. If CMake was not used to make OpenCV, then we need to write a new file named Find<name>.cmake file (i.e FindOpenCV.cmake), and put it in the directories listed in MAKE_MODULE_PATH. This is called as Find-module package. Now, in order to use this package in another project, we have to tell our new project, how to find this package. Packages are found with the find_package() macro. Both of this kind can be found using find_package() cmd.

1. Find-module Packages => These are the ones that have Find<name>.cmake file associated. This is the default package type that is assumed for any package (i.e CMake assumes that external package was not built with CMake). In order to use YARP package in new project called "hello", CMake searches the directories listed in CMAKE_MODULE_PATH for a file called Find<name>.cmake. If found, this macro is executed and it is responsible for finding the package.

FIND_PACKAGE(YARP) => searches for file named FindYARP.cmake in dir specified by CMAKE_MODULE_PATH 

2. Config-file Packages => These are the ones that were built with Cmake and have "<name>Config.cmake" or or <lower-case-name>-config.cmake files associated. This file is created in build dir selected by user, when CMake was used to build that package. In order to use YARP package in new project called "hello", we specify the location of this file by filling a cache entry called <name>_DIR (this entry is created by CMake automatically). If the file is found, it is processed to load the settings of the package (an error or a warning is displayed otherwise).

So, we set YARP_DIR to appr dir where this file is, and then use FIND_PACKAGE macro.

SET(YARP_DIR "$ENV{YARP_ROOT}") #We can also specify DIR directly as \usr\include instead of having env var
FIND_PACKAGE(YARP) => searches for file named YARPConfig.cmake or yarp-config.cmake in YARP_DIR specified above.

YARPConfig.cmake creates the entries YARP_LIBRARIES, YARP_INCLUDE_DIRS and YARP_DEFINES. To use these in a project, do:

  INCLUDE_DIRECTORIES(${YARP_INCLUDE_DIRS})
  ADD_DEFINITIONS(${YARP_DEFINES}) # optional
  ...
  TARGET_LINK_LIBRARIES(your_target_name ${YARP_LIBRARIES})

include:  include(file|module) => Load and run CMake code from a file or module. If a module is specified instead of a file, the file with name <modulename>.cmake is searched first in CMAKE_MODULE_PATH, then in the CMake module directory. 

ex: include(FindGSL) => searches for FindGSL.cmake and runs it.

 

 

1. find_module(Qt4 MODULE) =>

1. pkg_check_modules(<PREFIX> <MODULE>) invokes pkg-config, queries for <MODULE> and returns the result in variables which have names beginning with <PREFIX>_. Thus module is the name of the software you or pkg-config are looking for.

With "pkg-config --list-all"  (type this in build dir on linux shell cmd line) you get a list of all modules known by pkg-config. If your <MODULE> is not listed here, then  pkg_check_modules can never find it. If it does find it, use "pkg-config --modversion <MODULE>" to find what version of <MODULE> it's finding.

Ctest:

/usr/local/------

Now, to enable testing, we can add more lines in CMakeLists.txt :

cmake_minimum_required(VERSION 2.8 FATAL_ERROR)

project("My tutorial") 
enable_testing() #Enables testing for this CMake project. This should only be used in top level CMakeLists.txt. The main thing this does is enable the add_test() command.  
add_executable(hello hello.c)
add_test(HelloTest hello) => The enable_testing() function we added to our CMakeLists.txt adds the “HelloTest” target to our Makefile. Making the “HelloTest” target will run CTest which will, in turn, run all of our tests. In our case just the one.

 CPack:

-------

Design Reliability:

In chips, we have transistors and wires connecting them. They need to be functioning for 10 yrs or so. We run simulations to account for aging of transstors and wires.

BEOL=Back end of Line (metals and Vias),

FEOL = Front end of Line (transistors, everything below contact)

Transistor reliability: We measure many of these parameter affect for 10 yrs or 10K hrs of operation. FIT (failure in Time) rates are based on 10yrs of operation, so

  1. CHC: channel hot carrier: affects Transistor VT
  2. BTI: Bias Temperature instability: It degrades VT of Transistor during "ON" state at high temperature. It affects both NMOS and PMOS, though PMOS is affected more (reduces VT for NMOS, increases VT for PMOS => weaker tran).BTI happens when tran is ON, but starts fixing itself once tran is off.
    1. So, there are 2 BTI phase:
      1. Stress phase: here tran is ON, gate bias is max, causing max VT shift
      2. Relaxation phase: here tran is OFF, gate bias is min, reversing VT shift caused by stress phase. It's able to recover 20% of the VT shift caused by Stress phase. So, only 80% degradation of VT remains after Relaxation phase.
    2. Comparing BTI across tech:
      1. 180nm device => For PMOS which is contantly ON for 100K hrs @105C, with Vgs=1.8V, ΔVT = 10mv. 10mV is acceptable since there's already enough margin built into design.
      2. 10nm FinFet device => for NMOS, it's about 10mv, and PMOS is 50mv (assuming worst case 100% ON). At 1V applied across src/drn for 1nm device, Field is 1V/nm => 1KV/um, enough to break Si and SiO2 atoms. Better PBTI for finfet. NMOS BTI
    3. 2 kinds of BTI (-ve and +ve):
      1.  A. NBTI: -ve BTI, VT goes down. Here gate is at lower voltage than src, drn and subs. For tran to remain ON, this can only happen for PMOS. It is dominated by substrate interface traps. It's more severe than PBTI.
      2. B. PBTI: +ve BTI. VT goes up. Here gate is at higher voltage than src, drn and subs. For tran to remain ON, this can only happen for NMOS.  It is dominated by bulk HK trapped charges
  3.  HCI: same as CHC? FinFet HCI very challenging
  4. Self Heating: worse in Finfet as compared to planar, as fins surrounded by oxide.
  5. SEU : soft error vulnebility in srm cells, logic.
  6. ESD: ESD needed to protect gate oxide. In Finfet, Gate oxide breakdown is lower, and breakdown voltage of devices is also lower (=> FF more fragile). So, equiv ESD requires more area, bigger ESD devices.

 

Few more reliability notes:

 

  • Effective VT degradation = CHC + BTI
  • OverDrive capacity of Tran (i.e how much over voltage can they tolerate): For tran aging, stress voltages applied beyond rated voltages (1 V=rated voltage for 10nm NMOS/PMOS, stress voltage =1.2V), and higher temperatures. That's why thin oxide tran not used in IO. We make 2D plot of Voltage and temperature, and check the lifetime of tran, as well as VT shift of tran. We see that higher Voltage (30% higher than rated Voltage, so 1V rated tran stressed to 1.3V) and Higher Temp (> 50C), reduces lifetime of tran to couple of days. Very high voltages of >1.5V and Temp > 50C start causing VT shifts of 50mv or more (else it remains < 10mv). So, to assess Over Drive capability w/o getting the effect of VT shift, experiments limited to 30% voltage overdrive with Temp 50C.
  • HTOL stress (high temp operating life): High vol, high temp for 12-24 hrs gets same effect as aging.

 

 

Wire reliability:

Wires/Vias may suffer from reliability too. They may develop open/shorts due to multiple reasons.

  1. EM: lectro migration: Iavg/Irms/Ipeak calc.
  2. Self heating: happens for wires too.

 

Autotools:

We saw the make utility for building (compiling and generating executable) large programs. However, usually we write Makefile for compiiing programs for the system on which we are running make. i.e if I am working Linux OS, executable I generate is for Linux OS. However, that same executable will not work on Windows, and even on other flavors of Linux OS. This is because same compiler version, lib files, etc may not exist on Linux systems. So, we need different Makefile tailored for each platform. This will create too many Makefile each unique for each system.

To remedy this situation, we need to have a Makefile which will have #ifdef for different platform, and will generate executable differently for different platform (i.e if older version of gcc exists on some system, it may not support some flag. In that case, ifdef will help resolve such issues, and still let the Makefile build an executable). Having all these ifdef in Makefile and making sure that such pgm work on all arch is very tedious task. So, most of the software that is intended to be distributed for multiple platforms, use a tool called "Autotools". This tool automates the task of generating binary for different platforms.

Most of the Linux programs that we download, have their binary executable or source code. The programs which have their binary executable directly available, we do not need to do anything. We just run the executable and pgm starts running. However, programs which have their source code available, we usually need to run 3 steps as a end user to generate executable. This is called "GNU Build". Installer on your system unpacks the downloaded package (if tar.gz file, then use gunzip and tar, if .deb pkg, then use dpkg or apt, etc) and then runs these 3 steps:

  1. ./configure => analyzes your system to see what kind of programs and libraries you have, so it knows how to build the program best
  2. make => actual building is done using Makefile generated from above step (same way as we do using make with a Makefile)
  3. sudo make install => installs the pgm (puts pgm libs,binary,etc in appr dir with appr permissions). By default, it's put in /usr/local/ (bin in /usr/local/bin, lib in /usr/local/lib, etc)

These 3 steps are needed because over here Makefile gets generated differently for different platforms. ./configure generates a Makefile then make runs on this Makefile to generate executable, and then "install" puts generated executable in appropriate dir. We may never need to write a program that we distribute to other people, so you may wonder why learn Autotools. Reason is that most of the times we end up using these pgms, which requires us to run these steps. Having a brief understanding of this tool "Autotools" helps us when building (compiling and installing) 3rd party pgms on our system. Learning Autotools is a full time job in itself, so I'll just highlight few basic cmds with an example.

Full detailed tutorial for autotools here: https://www.lrde.epita.fr/~adl/autotools.html

Brief tutorial on this is: http://markuskimius.wikidot.com/programming:tut:autotools

Autotools is a collection of three tools:

  • Autoconf — This is used to generate the “configure” shell script. As I mentioned earlier, this is the script that analyzes your system at compile-time. For example, does your system use “cc” or “gcc” as the C compiler? Full Autoconf doc here: https://www.gnu.org/software/autoconf/manual/autoconf.html
  • Automake — This is used to generate Makefiles. It uses information provided by Autoconf. For example, if your system has “gcc”, it will use “gcc” in the Makefile. Or, if it finds “cc” instead, will use “cc” in the Makefile. Full automake docs here: https://www.gnu.org/software/automake/manual/automake.html
  • Libtool — This is used to create shared libraries, platform-independently. No need to know this as it's complicated topic for advanced users.

Autotools build process has some standard things in GNU build. Good to know these:

A. make options:

1. Std Makefile targets: make all, make install, make uninstall, make clean, make check=> To make targets when pkgs have been downloaded to your system

2. For making distribution: make dist (creates a tarball named *.tar.gz by collecting all src/other files, which is ready for distribution), make distcheck (to check the pkg for any errors/issues), make distclean.

3. staged installation. using DESTDIR, we can divert install step of "make install" to other dir than the ususal dir. Then we can choose and move files to whichever dir we want.

ex: make DESTDIR=~/scratch install

B. configure options: configure --help gives all options, few important ones are listed below.

1. Std Directory var: var=prefix (default is /usr/local). By chnaging value of this, we can put bin, lib, doc, etc in other dir.

ex: ./configure --prefix ~/user => puts bin "hello" in ~/usr/bin,/hello, etc.

2. Std configuration var: CC, CFLAGS, LDFLAGS, CPPFLAGS.

ex: ./configure CC=gcc3 .. => configure automatically chooses appr default values for these, but sometimes we may want to override defaults.

3. Parallel build tree: GNU build system has 2 trees: source tree and build tree. Source tree is the dir containing "configure" which has all src files. Build tree is dir where "./configure" is run creating object files and other intermediate files. Most of the times, we run "./configure" in same dir where configure is located, so source and build tree are same. But, if we want to keep our source files uncluttered from generated files, we can have build tree in separate by doing this:

ex: ~/.../top-dir-pkg (this is dir where you extracted files, and has configure script). "mkdir build", "cd build", run "../configure" and "make" in build dir. This keeps all generated files in build dir, keeping main source dir intact.

4. cross compilation: To generate binary for a different system that one where we are compiling the files. By default, binaries are generated for the same system, where we compile the files.

ex: ./configure --build=i68cpc --host=solaris => Here, build denotes our sytem, whereas host is the system for which we generate binaries. For binaries to get generated for host system, cross compiler has to exist on native system, else it will error out.

5. pgms can be renamed by using --program-prefix, --program-suffix (i.e instead of installing a pgm with name "tar", we can install it as "my-tartest, by using prefix=my-, suffix=test, to prevent overwriting "tar" that is already installed)

6.

Simple example of building a pkg: This ex shows how to build pkg like *.tar.gz from source files that can be distributed.

Autotools is installed by default. We can check version number of Autoconf/Automake by running autoconf/automake.

$ autoconf --version
autoconf (GNU Autoconf) 2.69

$ automake --version
automake (GNU automake) 1.13.4

$ autoreconf --version
autoreconf (GNU Autoconf) 2.69

ex1: write a C pgm (hello.c) that has gettimeofday function. We need to run autotools  in same dir as pgm hello.c. These are the steps for running autotools.  The whole goal of autotools is to generate 2 files: configure and Makefile.

0. write C pgm as below called hello.c:

#include <stdio.h>
#include <sys/time.h> // this is added on purpose, so that we can make this system dependent
 
int main(int argc, char* argv[])
{
   double sec;
   struct timeval tv;
 
   gettimeofday(&tv, NULL); // This function only exists in sys/time.h, so if that file doesn't exist, this will error out
   sec = tv.tv_sec;
   sec += tv.tv_usec / 1000000.0;
 
   printf("%f\n", sec);
 
   return 0;
}

1. autoconf + automake steps =>

  • Autoconf has a series of steps to generate configure script. configure is a very large bash script. Autoconf needs configure.ac as an i/p file to generate configure script.

    we can write configure.ac as below: AC_* and AM_* are M4 macros. AC_* are autoconf macros, while AM_* are automake macros.

    AC_PREREQ([2.69]) => Autoconf version number (optional)
    AC_INIT([hello-pkg], [1.0], [This email address is being protected from spambots. You need JavaScript enabled to view it.]) => pkg name, version, email addr for bug reporting
    AC_CONFIG_SRCDIR([hello.c])
    AM_INIT_AUTOMAKE([-Wall -Werror foreign]) => NOTE: this is automake macro (AM_*), not autoconf macro (AC_*). options inside are optional. We turn on all Warnings and report them as error by using -Wall and -Werror. -foreign allows us to proceed even w/o having files as README, AUTHORS, NEWS, etc. Else, automake will complain about these missing files and won't allow us to generate pkg. Also, if autoconf is run with this AM_* macro, it will error out as "undefined macro"
    AC_CONFIG_HEADERS([config.h]) => causes the configure script to create a config.h file gathering ‘#define’s defined by other macros in configure.ac. This config.h file can be included in hello.c file and then those defined strings in our program to make it portable.
    AC_PROG_CC => causes the configure script to search for a C compiler and define the variable CC with its name
    AC_CHECK_HEADERS([sys/time.h]) => checks for header files
    AC_CHECK_FUNCS([gettimeofday]) => checks for library func in src files
    AC_CONFIG_FILES([Makefile]) => list of all Makefiles that should be generated from Makefile.in file. If Makefile are in nested dir, provide all those here
    AC_OUTPUT => closing command that actually produces the part of the script in charge of creating the files registered with AC_CONFIG_HEADERS and AC_CONFIG_FILES (i.e config.h and Makefile)

  • Aytomake generates Makefile.in. It needs Makefile.am and configure.ac as i/p to generate Makefile.in.

    Makefile.am can be simple file specifying o/p binary file, and i/p C pgm as shown below.

    bin_PROGRAMS=hello
    hello_SOURCES=hello.c

2. autoreconf --install => With these 3 files above (hello.c, configure.ac, Makefile.am), we can now run autoconf on configure.ac to generate configure. Then we can run automake on Makefile.am and configure.ac to generate config.h.in and Makefile.in. But it will require lot of work to get it working. Autoreconf is a script that calls autoconf, automake, and a bunch of other commands in the right order. So, this is the preferred step instead of running autoconf and automake separately. This step creates configure, config.h.in, Makefile.in files. It also bunch of other files as install-sh, depcomp, missing, aclocal.m4 and dir autom4te.cache.

 3. configure => At his point build is complete. Steps 3 and 4 are what the user would run on any system that the package is downloaded to create executable. We run these steps here to check that everything runs OK. With 3 files (configure, config.h.in and Makefile.in) generated in step 2, running ./configure script (generated in step 2), creates Makefile (from Makefile.in) and config.h (from config.h.in). These 2 files have been created after probing the system, so these files are runnable on this system. There are also extra files created called config.status and config.log

4. make => running make generates executable hello (shows the actual steps). hello.c and hello will be the files generated by this step.

5. make install => We do not run this step as it will install binaries in appr dir, which we do not want on our system. Stpes 3,4,5 are run by folks downloading our pkg.

6. make distcheck => ceates final *.tar.gz distribution pkg as "hello-pkg-1.0.tar.gz"

Now that we have the final pkg, it can be dsitributed to anyone. However, our program is not yet portable for all systems, as there are function in our hello.c pgm that may not be present on some systems in the C library. config.h is the file that comes to our rescue here. It looks at all functions in pgm, and provides us with constants in form of "#define" that we can use to check if the system has that function in C library or not. For ex: looking in config.h, we see these lines:

/* Define to 1 if you have the `gettimeofday' function. */
#define HAVE_GETTIMEOFDAY 1

/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1

Now, in our C pgm, we can use these constants to check for the existence of these on that system. So, we modify our C pgm to make it portable.

Our modified C pgm looks like this:

#include <stdio.h>
#ifdef HAVE_SYS_TIME_H
#include <sys/time.h>
#else
#include <time.h>
#endif

int main(int argc, char* argv[])
{
   double sec;
 #ifdef HAVE_GETTIMEOFDAY
   struct timeval tv;
 
   gettimeofday(&tv, NULL);
   sec = tv.tv_sec;
   sec += tv.tv_usec / 1000000.0;
 #else
   sec = time(NULL);
#endif
   printf("%f\n", sec);
 
   return 0;
}

Now, since we modified our pgm, we need to rerun step 3 and 4 to make sure our pgm still compiles fine. Then we can run step 6 to create tar.gz that can be distributed.

----------

OPTIONAL: The steps below are alternate set of steps that are not recommended. But they are good to know, incase we do not want to run autoreconf, but instead plan to run autoconf and automake separately.

A. autoscan => generates configure.scan. It's a small file. It should look very similar to configure.ac file above. It has all autoconf macros  only(i.e AC_*). We will need to add automake macros to it (AM_*), Rename it as configure.ac to use it in flow above.

B. autoconf => uses configure.ac to generate configure. If needs config.h.in and Makefile.in . If we do not want to write config.h.in from scratch, we can use autoheader to generate config.h.in. Makefile.in contains very basic Makefile isntructions, which are used to generate Makefile.

C. autoheader => generates config.h.in. It just has few constants which are undefined.

D. automake =>  generates Makefile.in from Makefile.am.

E. aclocal => There will be lot of errors in automake step above. 1st set of errors will be Automake macros which aren't found in configure.ac. If we add these macros in configure.ac, then autoconf will freak out, since it doesn't know these macros. To remedy this, we provide defn of these macros in aclocal.m4. We run aclocal to creeate aclocal.m4 automatically with defn of all these macros in automake.

At his point, we have config.h.in and Makefile.in. So, configure script can run now, followed by make.

G. run configure: . ./configure => generates config.h from config.h.in, and Makefile from Makefile.in. config.h will look same as config.h.in, except that all constants are #define now. Makefile will look same as Makefile.in.

H. run make => Once Makefile is generated, we can run make. It uses Makefile to run 1st target "hello". make all => generates executable hello using rules in Makefile. Now we can run ./hello to get executable running.

 

GCC: GNU Compiler Collection

Before learning C or C++, we need to learn how to compile the C/C++ program. The program to compile C/C++ into machine code is call GCC. (GNU Compiler Collection). Very good pdf here (by Brian Gough) = https://tfetimes.com/wp-content/uploads/2015/09/An_Introduction_to_GCC-Brian_Gough.pdf

Installing GCC:

Check if gcc is installed by running "gcc -v" on your linux terminal.

gcc -v => shows "gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) " along with some other info.

GCC is installed by default on CentOS. However, on Linux Mint, you will get errors regarding various std lib not found, when trying to run GCC (even though gcc is installed). This is because not all the libs needed for gcc are installed. If you get error running gcc and gcc is already installed on your system, follow these steps ($ below represents the terminal prompt).

Debian based OS: (Linux Mint, Ubuntu etc): Run following 2 cmds:

$ sudo apt update => updates pkg data repository. Needed before you install anything

$ sudo apt install build-essential => build-essential package is a reference for all the packages needed to compile a Debian package. It generally includes the GCC/g++ compilers and libraries and some other utilities (as Make, etc).

Fedora based OS (RHEL, CentOS, etc): Run following 2 cmds: 

$ sudo yum makecache => makes sure that the yum cache is up to date with the latest metadata. (Not sure if we can use "sudo yum update" instead of this)

$sudo yum group install "Development Tools" => "Development tools" is a yum group which contains all pgms for compiling etc (gcc, cvs, rpm-build, etc). This installs all of those in 1 cmd.

 

Running "which gcc" shows that gcc is in path /usr/bin/gcc (binary file). Compiler "cc" used to be the default compiler in past, so usually there is soft link in /usr/bin/cc pointing to gcc, so that cc can be run as well.

We will explore gcc in more detail as we learn C and C++. Here are the basics with the help of a C/C++ pgm. C pgm need gcc to compile, while C++ require g++ compiler.

Compiling C pgm using gcc:

C pgm ex: write program hello.c as below

#include <stdio.h> => this file is in /usr/include/stdio.h

int main (void) {

  printf ("Hello, world!\n"); => printf is a function that is declared in stdio.h, so stdio.h had to be included. Only the declaration of function is done in stdio.h, actual body of function "printf" is itself stored in library /usr/lib/libc.a.

  return 0;

}

#include files:
-------------
2 versions of #include preprocessor directive. Full path, partial path or just name of file can be provided. If full path is provided, then 2 versions of #include have same effect, else they differ in how they search for the file.
1. #include <file_name> => system include. used for std header files. Here compiler searches for the file in std paths. Usually it's /usr/local/include (higher precedence) and /usr/include (lower precedence). We can provide full path of file here too, however that is not a good habit, as that file may not have same path on other systems, thereby making this pgm non portable. There is -I option that can be used for non-std path, which is discussed later.
2. #include "file_name" => user include. used for user defined header files. Here compiler first searches for the include file in the dir where your current source file resides. The current source file is the file that contains the directive #include "file_name". The compiler then searches for the include file according to the search order described above in version 1.

GCC options:

To compile pgm above, type:
gcc hello.c => This compiles hello.c pgm into an executable called a.out in same dir. Running ./a.out will print "Hello, world" on screen. # directive isntructs compiler to include stdio.h file at appropriate points. That is why we don't need to explicitly compile this file.

gcc -Wall -v hello.c -o hello => -o specifies that output executable file should be named hello instead of a.out. -Wall turns on all warnings (recommended to always use it). We can turn on specific warnings by using -Wcomment, -Wformat, etc (or even more warnings by using -W in addition to -Wall) .-v shows details about various paths, options used.

Producing machine language executable is a 2 step process, when multiple files are involved. First we create an compiled object file for each source file, and then a linker program (called ld but it's invoked automatically by gcc) links all these compiled object files to produce an executable a.out. An object file contains machine code where any references to the memory addresses of functions (or variables) in other files are left undefined.This allows source files to be compiled without direct reference to each other. The linker fills in these missing addresses when it produces the executable.

steps:

1. gcc -Wall -c main.c => If we use option -c, then instead of generating executable file, object file called main.o is genberated. Here object file with same name as source file is created by default (so main.c creates an object file main.o). Similarly, we create object files for all other files. When creating object files, compiler just notes any unresolved symbols and leaves the addr "blank" for that symbol/function.

2. gcc -Wall -c other.c => generates other.o

3. gcc main.o other.o -o hello => this step calls linker ld, which links all object files to create an executable. Now, ./hello can be run. Order is important here. Files are searched from left to right, so files which have functions that are called by other files should appear last. So, if main.c has a function my_func defined in other.c, then main.o should be put before main.o.

Instead of running the 3 cmds separately, we can also run it in 1 cmd as follows:

gcc main.c hello.c hello => produces executable hello

Linking with external libraries:

A library is a collection of precompiled object files which can be linked into programs. Libraries are typically stored in special archive files with the extension‘.a’, referred to as static libraries. They are created from object files with a separate tool, the GNU archiver ar, and used by the linker to resolve references to functions at compile-time. The standard system libraries are usually found in the directories ‘/usr/local/lib’ (higher precedence) and ‘/usr/lib’ (lower precedence). On 64 bit platforms, additional lib64 dir are also searched.

C std lib: /usr/include/stdio.h and few other *.h has all std header files (which have function declaration), while /usr/lib/libc.a is the C std lib which has all the functions defined in C std. We just include the header files in C pgm. Then the std C lib is linked by default for all C pgms.

C math lib: /usr/include/math.h has all std header files (which have function declaration for math functions as sqrt), while /usr/lib/libm.a is the C math lib which has all the math functions. This lib is not linked by default, even if we include math.h in the C pgm. Compiler option -lNAME (small letter "l" (as in love) with no space b/w l and NAME) will attempt to link object files with a library file ‘libNAME .a’ in the standard library directories. So, to link math lib, we should use "-lm" (that links libm.a from std dir which is /usr/lib/). To link more lib, we'll need -lNAME for each of them. Instead of -lm, we can also provide the full path of file as /usr/lib/libm.a on cmd line of gcc.

 The list of directories for header files is often referred to as the include path and the list of directories for libraries as the library search path or link path.

 When additional libraries are installed in other directories it is necessary to extend the search paths, in order for the libraries to be found.The compiler options ‘-I’ (capital I as in India)and ‘-L’ (captal L as in Love) add new directories to the beginning of the include path and library search path respectively.

ex: gcc -Wall -I/opt/gdbm/include -L/opt/gdbm/lib dbmain.c -lgdbm (here non std gdbm pkg is installed in /opt/gdbm. gdbm.h is in /opt/gdbm/include/gdbm.h, while libgdbm.a is in /opt/gdbm/lib/gdbm.a)

There are environment variables also which can be set instead of -I and -L options above.

1. include path: var C_INCLUDE_PATH (for C header files), CPLUS_INCLUDE_PATH (for C++ header file)

2. Static Lib search path: var LIBRARY_PATH

These var can be set on cmdline, or be put in .bashrc file, so that they take affect all the time.

ex: add these in .bashrc in home dir.

C_INCLUDE_PATH=.:/opt/gdbm-1.8.3/include:/net/include:$C_INCLUDE_PATH => adds current dir (due to . in front) and other paths to C_INCLUDE_PATH if it had any.

LIBRARY_PATH=.:/opt/gdbm-1.8.3/lib:/net/lib:$LIBRARY_PATH => adds current dir and other paths to LIBRARY_PATH if it had any.

export C_INCLUDE_PATH; export LIBRARY_PATH => export cmd is needed so that these var can be seen outside of current shell by other pgms as gcc.

So far, we have been dealing with static libraries. There is concept of shared libraries explained nicely in pdf book. Dynamic linking of these shared libraries is done at run time, so executable file (a.out) is smaller in size (as a.out doesn't contain full object file of function in .a file). Instead it keeps a small table that tells it where to get it from. OS takes care of this by loading a single copy of shared lib in dram memory, and providing a pointer to that shared lib whenever a.out requests access to shared lib. Instead of .a extension, they have .so extension, and reside in same dir where .a files reside. By default, .so files will be linked instead of .a files if .so files are present. If .so files are in non std path, then we either need to provide full path to .so file on cmd line, or need to add this 3rd var also:

3. Dynamic lib search path: var LD_LIBRARY_PATH

We can force compiler to do static linking only by using option -static.

C language standards:

original C language std are called ANSI/ISO C std (called c89 and c99). Then GNU added extensions to language called as GNU std (called as gnu89 and gnu99). By default, gcc compiles GNU C pgm. That means it uses gnu C lib (glibc). However, if we want strict ANSI/ISO C pgm, we can compile with -ansi or -std c99 option.

 Preprocessor:

# statements in C. #defiine and #ifdef ... #endif are used to compile only desired sections of C code. Instead of using #define in C pgm (which will require changing C pgm), we can define it on cmd line using -DNAME (i.e for #ifdef TEST ... #endif, we can do -DTEST which is equiv to #define TEST). To define value to var, we can do -DNUM=23 (equiv to #define NUM 23), or DMSG="My Hero", etc.

 Optimization level:

different opt levels are supported by gcc. -O0 is level 0 opt, and is the default. -O1, -O2 and -O3 refers to higher levels of code opt.

 Platform specific options:

GCC produces executable code which is compatible with all the processors in the x86 family by default if it's running on x86 system —going all the way back to the 386. However, it is also possible to compile for a specific processor to obtain better performance.

gcc -march=pentium4 => produces code that is tuned for pentium4, so may not work on all x86 processors. Better to not use this option, as it provides a little speed improvement. Similarly there are options for powerpc, sparc, dec alpha processors.

gcc -m32 generates 32 bit code on 64bit AMDx86-64 systems. Not using -m32 will produce 64 bit code by default.

other options:

gcc --help

gcc --version

gcc -v test.c => verbose compilation, shows exact seq of cmds used to compile and link. Shows full dir paths used to search header files and libs.

Compiling C++ pgm using g++:

C++ pgm ex: write program hello.cc as below

#include <iostream>

int main () {

   std::cout << "Hello, world!" << std::endl; //similar to printf func of C

   return 0;

}

compile: g++ -Wall hello.cc -o hello => here we used g++ for compiling C++ pgm. We could have used gcc too as it would compile all files ending in .cc, .C, .cxx or .cpp as C++ pgm.  The onky problem that may happen when using gcc to compile C++ files, is that the appropriate C++ lib may not get linked (*.o files produced by g++ can't be linked using gcc). It' always preferable to use g++ for compiling C++ pgm, and gcc for C pgm. g++ has exactly same options as gcc.

C++ std lib: The C++ standard library ‘libstdc++’ supplied with GCC provides a wide range of generic container classes such as lists and queues, in addition to generic algorithms such as sorting.

Compiler related tools:

1. GNU archiver : called as "ar", it combines a collection of object files into a single archive file, also known as a library.

ar cmd: ar cr libfn.a hello.o bye.o =>  creates a archive from 2 simple object files. cr=create and replace

ar t libfn.a => lists all object files in archive. Here it lists hello.o and bye.o

gcc -L. main.c -lfn -o main => This lib archive libfn.a can be used like any other static lib. -L. just adds . to lib search path (assuming we generated libfn.a in current dir)

2. grpof: gnu profiler for measuring performance of pgm.

3. gcov: gnu coverage tool analyzes coverage of pgm = how many times each line of pgm is run during execution

Compiler steps: Running gcc/g++ involves these 4 steps. These are all run behind the scenes when running gcc/g++, but can be run separately too.

1. preprocessing of macros: preprocessor expands all amcros and header files.

ex: gcc hello.c > hello.i => hello.i contains source code with all macros expanded

2. assembly code generation: assembly code is then generated. It still has call to extenal functions.

ex: gcc -S hello.i => hello.s is generated which has assembly code

3. assembler: converts assembly language into machine code and generate an object file. Addr of External functions still left undefined to be filled in by linker

ex: as hello.s -o hello.o

4. Linking: Any external functions from sytem or C run time lib (crt) are linked here.

ex: ld -dynamic-linker /usr/.../.so /../crt1.o hello.o ...  => All these object files linked together (with proper addr of func called)

ex: gcc hell.o -o hello => this gcc cmd invokes linker automatically when generating an executable from object files

Examining Compiled Files:

ex: file a.out => shows details of file a.out, whether it's ELF format, 32/64 bit, which processor it was compiled for (INTEL 80386, etc), dynamic/static link, and whether it contains a symbol table.

nm a.out => this shows location of all var and func used in exectable. T against a func name indicates func is defined in object file, while U indicates undefined (may be because it's going to be dynamically linked at run time, or we need to link that file having that func with this executable)

ldd a.out => This shows list of all shared lib, that are to be linked at runtime. It shows all dynamic lib (usually libc.so, libm.so), as well as dynamic loader lib (ld-linux.so)

 --------------------

 

graywolf is fork of Timberwolf:

TimberWolf doc is here: http://opencircuitdesign.com/qflow/archive/TimberWolf.pdf

To install Timberwolf, we need to install couple other software:

A. CMake => CMake is a popular alternative to autotools. It is usedin many open-source projects including large ones such as KDE, LLVM and Blender. See CMake section for more details on cmake.

Steps for downloading cmake:

1. dwnload cmake from here: https://cmake.org/download/ (cmake-3.14.0-rc2.tar.gz)

2.  extract .gz file, and goto dir "cmake-3.14.0-rc1). Now run these 3 steps: 1. /bootstrap 2. make 3. sudo make install

B. GSL => GNU Scientific Library (libgsl): this is used in C pgms to call many scientific functions.

1. downlaod GSL from here: https://www.gnu.org/software/gsl/ 2.goto gnu ftp mirror link and download gsl-latest.tar.gz (currently latest is pointing to version 2.5)

2. extract .gz. cd to gsl-2.5 dir. Now run these 3 steps: 1. ./configure 2. make 3. make install

3. We should see libgsl.so and libgslcblas.so shared lib in /usr/local/lib dir. Also make sure env var "LD_LIBRARY_PATH" is set to ":/usr/loacal/lib/" (assuming gsl was installed in defau;t pat). echo $LD_LIBRARY_PATH => ://usr/local/lib/ . All header files for gsl lib will be in /usr/local/inlcude/gsl

Now, to test that gsl is installed correctly, we can write a simple C test program which include one of the GSL functions, and see if works. Write a pgm named gsl_test.c

#include <gsl/gsl_sf_bessel.h> //this header file dir is in /usr/local/include dir
int main (void) {
    double x = 15.0;
    double y = gsl_sf_bessel_J0 (x);
    printf ("J0(%g) = %.18e\n", x, y);
    return 0;
}

Run:  gcc gsl_test.c -lgsl -lgslcblas => This should create a.out, and running a.out should produce bessel output.

-----

 Once above 2 software are installed, we install Timberwolf as follows:

1. Download graywolf from here: https://github.com/rubund/graywolf.

2. Download zip file, "graywolf-master.zip". Unzip it, and you should have a dir "garywolf-master". cd to that dir, and read thru README.md. It has instructions for installing it. Run these steps:

!. cd graywolf-master

II. mkdir buid

III. cd build

IV. cmake .. => NOTE: cmake is used to build this, instead of traditional GNU Autotools

This runs CMakeLists.txt in dir graywolf-master. At this stage, you may get an error:

--   No package 'gsl' found
CMake Error at CMakeLists.txt:15 (MESSAGE):
  The development files for the GNU Scientific Library (libgsl) are required
  to build graywolf.

This happens, since pkg_check_modules is not able to find GSL package, even though it's installed in std location.

To fix this, modify CMakeLists.txt in dir graywolf-master. comment out line "pkg_check_modules(GSL gsl)" and replace it with "include(FindGSL)". This will allow cmake to find GSL pkg.

#pkg_check_modules(GSL gsl)

include(FindGSL)

Once done, cd to build dir, and run "cmake .." once again. This time it should run fine:

[graywolf-master/build]$ cmake ..
-- Configuring done
-- Generating done
-- Build files have been written to: /home/Ajay/Downloads/graywolf-master/build

V. make => now run make, Last few lines on screen look something like this:

[100%] Built target mc_compact
Scanning dependencies of target run
[100%] Generating show_flows
[100%] Built target run

VI. sudo make install => this will install various files of this software in /usr/local/lib and /usr/local/bin

VII. make test => This will run 6 tests, but 1 of them fails.

The following tests FAILED:
      5 - map9v3-twmc (Failed)
Errors while running CTest
make: *** [test] Error 8

VIII. CTEST_OUTPUT_ON_FAILURE=1 make test => this step is optional. running this produces more verbose o/p for test 5 to help us debug.

iX: now cd to any dir, (cd /home/) type "graywolf" on terminal, and it should run it. Add steps on running it => FIXME