\documentclass[letterpaper]{article}
\usepackage{graphicx}

%% Set the \pdfpagewidth and \pdfpagelength, ONLY if we're using pdflatex.
\usepackage{ifthen}
\ifx\pdfoutput\undefined
\else
\setlength\pdfpagewidth{8.5in}
\setlength\pdfpageheight{11in}
\fi

%% Set up some margins that waste less paper :-)
\topmargin          = -0.5in
\oddsidemargin      =  0.0in
\textwidth          =  6.5in
\textheight         =  9.0in

\newcommand{\code}[1]{\vspace{3pt}\\\makebox[\textwidth]{\tt #1}\vspace{3pt}}
\newcommand{\codeleft}[1]{\vspace{3pt}\\\makebox[\textwidth][l]{\tt #1}\vspace{3pt}}

\begin{document}

\title{The dbgroup-utils Build Guide}
\author{Chavdar Botev (cbotev@cs.cornell.edu) \and Tony Novak (afn@cs.cornell.edu)}

\maketitle

\section{Building dbgroup-utils on Linux}

\subsection{Introduction}

The build process on Linux (and other flavors of Unix) is based on the GNU {\tt
make} suite. The process is defined by a set of files named "Makefile", one for
each directory where some kind of processing needs to be done. The Makefiles
contain a set of rules which describe how the build targets can be created. For
more information, refer to the man pages for make or read the online manual at
http://www.gnu.org/software/make/manual/make.html.

Although, creating Makefiles is sufficient for describing the build process,
this is often tedious and involves a significant amount of duplication becuase
the build process is similar for most of the directories. Moreover, the actual
build process may depend on the particular configuration of the computer system
where the build is to be performed. Therefore, it is often helpful to use tools
to automate the process of creation of Makefiles. 

Such kind of tools are contained in the GNU {\tt autotools} package which is
used by dbgroup-utils. In particular, we use two main tools: {\tt automake} and
{\tt autoconf}. The entire process is shown on figure
\ref{fig:lin-build-process}. 

\begin{figure}[tbh]
\begin{center}
\includegraphics[height=15cm]{linux-build-process.eps}
\end{center}
\caption{\label{fig:lin-build-process}Linux Build Process}
\end{figure}

The result of the entire process is shown as "Distribution files" at the bottom
of the figure. It is produced by the {\tt make} program using the per-directory 
Makefiles. The Makefiles are generated by the {\tt configure} script which is
created by the {\tt autoconf} program. The {\tt configure} script read the
Makefile.in files created by {\tt automake}. {\tt automake} uses the
user-supplied Makefile.am templates to generate the Makefile.in files. Both {\tt
automake} and {\tt autoconf} read the configure.ac file. The latter contains a
sequence of m4 macros and shell commands whose purpose is to test the
configuration of the computer system where the build will be performed. It
checks for necessary libraries and tools and specifies the Makefiles to be
generated by the {\tt configure} script. The m4 macros are defined in the
aclocal.my file. This file can be generated using the {\tt aclocal} program.

\subsection{Preparing a fresh copy of dbgroup-utils}

The preparation of a fresh copy of the repository  closely follows the process
described on figure \ref{fig:lin-build-process}.

\begin{itemize}
\item {\tt ./platforms/Linux/scripts/prepare-libs.sh} to create the necessary 
	  symbolic links for the libraries in {\tt platforms/Linux/extlib},
\item {\tt aclocal} to generate {\tt aclocal.m4} (needed by {\tt autoconf}),
\item {\tt automake -a} to generate some scripts needed by the automake program
	  (the -a switch) and then to create a {\tt Makefile.in} file from each
	  {\tt Makefile.am} template.
\item {\tt autoconf} to generate the {\tt configure} script, and finally
\item {\tt ./configure} to generate a {\tt Makefile} for each {\tt Makefile.in} 
	generated earlier by {\tt automake}.
\end{itemize}

\subsection{Using {\tt make} to build dbgroup-utils}

The simplest usage is using only 

\begin{verbatim}
	make
\end{verbatim}

or the equivalent

\begin{verbatim}
	make all
\end{verbatim}

Run this command from the root dbgroup-utils directory. This will compile all
source files, create the directory structure of distrib/ and copy all
distribution files to distrib/. Since {\tt make} echos all commands it runs on
the console, the screen can become very cluttered. To overcome this problem, one
can use the {\tt -s} option, which causes {\tt make} to be silent:

\begin{verbatim}
	make -s
\end{verbatim}

This is especially useful for finding warnings caused by the build process which
otherwise get lost in the long stream of {\tt g++} invocations.

One can decide to build only certain libraries and/or executables. In this case,
a command like 

\begin{verbatim}
	make build-<target>
\end{verbatim}

where "target" is the
name of the executable or library. This will build only the specified
executable/library. For example. 

\begin{verbatim}
	make build-corelib
\end{verbatim}

will build only the corelib library. When using this syntax, one must make sure
that all local libraries that the built target depends on have already been
built. Otherwise, the compiler will not find the necessary header or library
files and will report an error.

Another usefult command is

\begin{verbatim}
	make docs
\end{verbatim}

This will build the documentation for all libraries and executables. As above,
one can use

\begin{verbatim}
	make docs-<target>
\end{verbatim}

to build only the documentation for a specific target.


The command 

\begin{verbatim}
	make apidocs
\end{verbatim}

will build the API documentation for all libraries in dbgroup-utils. The command

\begin{verbatim}
	make apidocs-<target>
\end{verbatim}

will build the API documentation only for a specfic library.

\subsection{Common Tasks}

\subsubsection{Creating a repository}
\label{sec:create-rep-linux}

\begin{enumerate}
	\item Create a directory for the new project (e.g. {\em new\_proj}).
	\item Check out dbgroup-utils
	\item Copy all files from the root dbgroup-utils directory to {\em new\_proj/} .
	\item Edit the standard GNU files like {\em README}, {\em COPYIN}G, etc. in {\em new\_proj/} .
	\item Copy the {\em make\_templates/} directory from dbgroup-utils to {\em new\_proj/} .
	\item Copy the {\em platforms/} directory from dbgroup-utils.
	\item Add any external libraries to {\em new\_proj/platforms/}.
	\item Edit {\em new\_proj/configure.ac}
		\begin{enumerate}
			\item Edit the repository initialization - add the project name and
				  version to the {\tt AC\_INIT} and {\tt AM\_INIT\_AUTOMAKE}
				  macros (see Figure \ref{fig:lin-conf-init}).
 			\item Add checks for external libraries/tools (see Figure
				  \ref{fig:lin-conf-checklibs})
			\item Add target libraries/executables (see section
				  \ref{sec:add-target-linux})
		\end{enumerate}
	\item Run {\tt aclocal \&\& automake -a \&\& autoconf \&\& ./configure}
\end{enumerate}

\begin{figure}[tbh]
\begin{center}
\includegraphics{configure.ac-init.eps}
\end{center}
\caption{\label{fig:lin-conf-init}Project initialization in configure.ac}
\end{figure}

\begin{figure}[tbh]
\begin{center}
\includegraphics{configure.ac-checklibs.eps}
\end{center}
\caption{\label{fig:lin-conf-checklibs}Checks for external libraries configure.ac}
\end{figure}

\subsubsection{Creating a new library/executable target}
\label{sec:add-target-linux}

We will describe the case of adding a new library. Adding new executables is
similar.

\begin{enumerate}
	\item Create a top-level directory for the library(e.g. {\em mylib}).
	\item Create {\em mylib/src}, {\em mylib/doc}, {\em mylib/tests}
	\item Edit configure.ac
		\begin{enumerate}
			\item Define the library name and directory by adding new variables
				  as shown on Figure \ref{fig:lin-conf-addlib}.
 			\item Add library to the library list ({\tt LOCAL\_LIBS})  (see Figure
				  \ref{fig:lin-conf-addlib})
			\item List the {\em Makefile}s to be generated
				  (see Figure \ref{fig:lin-conf-files})
		\end{enumerate}
	\item Copy {\em make\_templates/Makefile.proj.am} to {\em mylib/Makefile.am} 
	\item Copy {\em make\_templates/Makefile.lib.am} to {\em
		  mylib/src/Makefile.am}
	\item Replace library-specific settings in the {\em Makefile.am}: set the 
		  values of the {\tt TARGET\_NAME} and {\tt DIR\_NAME} variables as 
		  shown on Figure \ref{fig:lin-make-lib-change}.
	\item Copy {\em make\_templates/doxy.cfg.in} to {\em mylib/}
	\item Edit the {\em doxy.cfg.in} file and set the library-specific
		  variables {\tt PROJECT\_NAME} and {\tt OUTPUT\_DIRECTORY} (Figure \ref{fig:lin-make-doxy-change}).
	\item Rerun {\tt automake \&\& autoconf \&\& ./configure} from the root
		  directory for the repository.
\end{enumerate}

\begin{figure}[tbh]
\begin{center}
\includegraphics{configure.ac-addlib.eps}
\end{center}
\caption{\label{fig:lin-conf-addlib}Library name and directory variables in 
		 configure.ac}
\end{figure}

\begin{figure}[tbh]
\begin{center}
\includegraphics{configure.ac-createfiles.eps}
\end{center}
\caption{\label{fig:lin-conf-files}Creation of library Makefiles}
\end{figure}

\begin{figure}[tbh]
\begin{center}
\includegraphics{make.lib-change.eps}
\end{center}
\caption{\label{fig:lin-make-lib-change}The {\tt TARGET\_NAME} and {\tt DIR\_NAME} variables}
\end{figure}

\begin{figure}[tbh]
\begin{center}
\includegraphics{doxy.cfg-change.eps}
\end{center}
\caption{\label{fig:lin-make-doxy-change} Doxygen configuration customization}
\end{figure}


\subsubsection{Adding a new source directory}
\label{sec:add-dir-linux}

\begin{enumerate}
	\item Create a new source directory (e.g. {\em newdir}).
	\item Copy {\em make\_templates/Makefile.dir.am} to {\em newdir/Makefile.am} 
	\item Replace the library-specific setting of {\tt DIR\_NAME} in the 
	{\em Makefile.am} (see Figure \ref{fig:lin-make-dir-change})
	\item Add the directory to the parent’s directory {\em Makefile.am} by
	appending the name of directory to the value of the {\tt SUBDIRS} variable
	(see Figure \ref{fig:lin-make-lib-change}). 
	\item Add the {\em Makefile.am} file from the new directory to the top-level 
	{\em configure.ac} file (see Figure \ref{fig:lin-conf-files}.
	\item Rerun {\tt automake \&\& autoconf \&\& ./configure} from the root
		  directory for the repository.
\end{enumerate}

\begin{figure}[tbh]
\begin{center}
\includegraphics{make.lib-change.eps}
\end{center}
\caption{\label{fig:lin-make-dir-change}The {\tt DIR\_NAME} variables}
\end{figure}


\subsubsection{Removing a source directory}
The process for removing directories is analogous.

\subsubsection{Adding a new source file}

Adding or removing files from the source tree requires no modification to any of 
the {\em Makefile}s. The build process will automatically discover any changes
in the files.  All source files will automatically be compiled and linked.

\section{Building Quark on Windows}

The build process on Windows is much less ``standard'' than on Linux, mainly because we wanted a flexible, automated system that could nonetheless work with Microsoft Visual Studio .NET\footnote{Currently, we {\em only} support Visual Studio .NET 2003, and no other version of Visual Studio or Visual C++.}.  Therefore, there is a set of batch files, scripts, and utilities, in the {\tt platforms/win32} directory, to generate the Visual Studio project files.

To generate the project file, which you'll have to do after a fresh checkout, simply double-click the {\tt configure.bat} file.  It will generate several project ({\tt .vcproj} files) and a solution file, {\tt quark.sln}; when it is finished, it will say:
\code{==> Finished successfully! <==}
\code{==> Set up for Visual Studio .NET 2003 (Visual C++ 7.1) <==}
\codeleft{Press any key to continue . . .}

At this point, press the proverbial Any Key, and double-click on {\tt quark.sln} to open the Solution.  Now, simply open the {\em Build} menu and choose {\em Build Solution}, and the Quark system will be built.

\subsection{Adding files and directories}

The build scripts automatically find all source and header files {\em anywhere} under the top-level directories (currently {\tt qe}, {\tt utils}, {\tt exceptions}, {\tt xmldb}, {\tt clp}, {\tt server}).  Therefore, if you add any files or subdirectories, as long as they are descendants of one of these directories, you should have to do nothing to get the files to be included in the project.  {\bf However}, if you add/remove a directory, you should make sure to follow the instructions in section \ref{sec:add-dir-linux} so that your changes will be reflected in the Linux build process!

Finally, although the scripts which generate the project files automatically find new files and directories, they will {\bf not} be automatically included in your {\em existing} project files (and therefore will not be compiled or linked).  So when you add a new file to Quark, you should either manually add the source file to your project in Visual Studio, or just re-run {\tt configure.bat} (make sure to close Visual Studio first; otherwise it will fail to re-create {\tt quark.sln}).  Similarly, when you do a {\tt cvs update}, it's generally a good idea to re-run {\tt configure.bat} in order to ensure that your project files include all files that have been recently added by others.

\subsection{Gory details}

{\centering \bf [ Should this possibly go into a different document? ]}

As mentioned above, there is no extra work required in order to add new files or directories as long as they are under one of the top-level directories.  However, if you need to add a directory at the top level (directly under {\tt src}) there is some extra work involved.

Each of the top-level directories yields a separate Visual C++ project file; the file is either a {\em library} or an {\em executable}.  The library projects each produce a static library; and each of the executable projects is linked against {\em all} the libraries.  So, for example, the {\tt clp} project file produces an executable, {\tt clp.exe}, which is built from all the sources in the {\tt clp} directory linked against the following libraries: {\tt qe.lib}, {\tt utils.lib}, {\tt exceptions.lib}, {\tt xmldb.lib}.  So, when you add a top-level directory, it will either be a library (if it's common code that's used in several different applications), or an executable (if it's an application that uses the Quark API).

So how do you add your new directory to the Windows build process?  First, you'll need to edit the {\tt Makefile} in the {\tt platforms/win32} directory.  Add the name of the new directory to either the {\tt BINS} or {\tt LIBS} variable depending on the type of project it is.  Finally, you may wish to customize what the project file for this project contains.  The project files are pieced together from template files in the {\tt templates} directory.  Each project file consists of five sections:

\vspace{-1.5\baselineskip}
\begin{quotation}
\noindent
\begin{description}
\sloppy
\item [inc1 (Header).] This contains the beginning of the project file; at the very least, the {\tt $<$VisualStudioProject$>$} and {\tt $<$Files$>$} tags should be opened here.
\item [Source File List.] This automatically-generated section contains the source files, organized by directory structure.
\item [inc2 (Intermediate Tags).] This contains any tags required between the source file and header file lists.  By default, it closes the {\tt Source Files} filter tag, and opens a new {\tt Header Files} filter tag.
\item [Header File List.] This automatically-generated section contains the header files, organized by directory structure.
\item [inc3 (Footer).] This should close any tags opened by the header.
\end{description}
\end{quotation}

Each of these sections can be specified at several different levels of granularity.  Each project file is built from the following files concatenated in order, where {\tt NAME} is the name of the project (e.g. {\tt clp}) and {\tt WHICH} is the type of project, either {\tt \verb$_BIN$} or {\tt \verb$_LIB$}.  Any files that are missing are simply skipped (thus some project files may use the standard templates, while others have more complex definitions).

\begin{enumerate}
\addtolength\itemsep{-\baselineskip}\addtolength\itemsep{6pt}
\item {\tt \verb$_DEFAULT$.vcproj.inc1}
\item {\tt WHICH.vcproj.inc1}
\item {\tt NAME.vcproj.inc1}
\item {\em Source File List}
\item {\tt \verb$_DEFAULT$.vcproj.inc2}
\item {\tt WHICH.vcproj.inc2}
\item {\tt NAME.vcproj.inc2}
\item {\em Header File List}
\item {\tt \verb$_DEFAULT$.vcproj.inc3}
\item {\tt WHICH.vcproj.inc3}
\item {\tt NAME.vcproj.inc3}
\end{enumerate}

Finally, each of these template files has a number of macros, such as {\tt \verb$__ADDITIONAL_DEPENDENCIES__$} and {\tt \verb$__VCVERSION__$}, which are expanded by the {\tt Makefile}.  For more details on this feature, refer to the {\tt Makefile}.

\end{document}
