<title>The Fossil Build Process</title>
<h1>1.0 Introduction</h1>
The build process for Fossil is tricky in that the source code
needs to be processed by three different preprocessor programs
before it is compiled. Most users will download a
[http://www.fossil-scm.org/download.html | precompiled binary]
so this is of no consequence to them, and even those who
want to compile the code themselves can use one of the
[./build.wiki | existing makefiles].
So must people do not need to be concerned with the
build complexities of Fossil. But hard-core developers who desire
a deep understanding of how Fossil is put together can benefit
from reviewing this article.
<a name="srctour"></a>
<h1>2.0 Source Code Tour</h1>
The source code for Fossil is found in the
[/dir?ci=trunk&name=src | src/] subdirectory of the
source tree. The src/ subdirectory contains all code, including
the code for the separate preprocessor programs.
Each preprocessor program is a separate C program implemented in
a single file of C source code. The three preprocessor programs
are:
1. mkindex.c
2. translate.c
3. makeheaders.c
Fossil makes use of [http://www.sqlite.org/ | SQLite] for on-disk
storage. The SQLite implementation is contained in three source
code files that do not participate in the preprocessing steps.
These three files that implement SQLite are:
4. sqlite3.c
5. sqlite3.h
6. shell.c
The sqlite3.c and sqlite3.h source files are byte-for-byte copies of a
standard [http://www.sqlite.org/amalgamation.html | amalgamation].
The shell.c source file is code for the SQLite
[http://www.sqlite.org/sqlite.html | command-line shell] that is used
to help implement the [/help/sqlite3 | fossil sql] command. The
shell.c source file is also a byte-for-byte copy of the
shell.c file from the SQLite release.
The TH1 script engine is implemented using files:
7. th.c
8. th.h
These two files are imports like the SQLite source files,
and so are not preprocessed.
The VERSION.h header file is generated from other information sources
using a small program called:
9. mkversion.c
The builtin_data.h header file contains the definitions of C-language
byte-array constants that contain various resources such as scripts and
images. The builtin_data.h header file is generate from the original
resource files using a small program called:
10 mkbuiltin.c
The src/ subdirectory also contains documentation about the
makeheaders preprocessor program:
11. [../src/makeheaders.html | makeheaders.html]
Click on the link to read this documentation. In addition there is
a [http://www.tcl.tk/ | Tcl] script used to build the various makefiles:
12. makemake.tcl
Running this Tcl script will automatically regenerate all makefiles.
In order to add a new source file to the Fossil implementation, simply
edit makemake.tcl to add the new filename, then rerun the script, and
all of the makefiles for all targets will be rebuild.
Finally, there is one of the makefiles generated by makemake.tcl:
13. main.mk
The main.mk makefile is invoked from the Makefile in the top-level
directory. The main.mk is generated by makemake.tcl and should not
be hand edited. Other makefiles generated by makemake.tcl are in
other subdirectories (currently all in the win/ subdirectory).
All the other files in the src/ subdirectory (79 files at the time of
this writing) are C source code files that are subject to the
preprocessing steps described below. In the sequel, we will call these
other files "src.c" in order to have a convenient name. The reader
should understand that whenever "src.c" or "src.h" is used in the text
that follows, we really mean all (79) other source files other than
the exceptions described above.
<h1>3.0 Automatically generated files</h1>
The "VERSION.h" header file contains some C preprocessor macros that
identify the version of Fossil that is to be build. The VERSION.h file is
generated automatically from information extracted from the "manifest",
"manifest.uuid", and "VERSION" source files in the root directory of the
source tree.
(The "manifest" and "manifest.uuid" files are automatically generated and
updated by Fossil itself. See the [/help/setting | fossil set manifest]
command for additional information.)
The VERSION.h header file is generated by
a C program: src/mkversion.c.
To run the VERSION.h generator, first compile the src/mkversion.c
source file into a command-line program (named "mkversion.exe")
then run:
<blockquote><pre>
mkversion.exe manifest.uuid manifest VERSION >VERSION.h
</pre></blockquote>
The pathnames in the above command might need to be adjusted to get the
directories right. The point is that the manifest.uuid, manifest, and
VERSION files
in the root of the source tree are the three arguments and
the generated VERSION.h file appears on standard output.
The builtin_data.h header file is generated by a C program: src/mkbuiltin.c.
The builtin_data.h file contains C-langauge byte-array definitions for
the content of resource files used by Fossil. To generate the
builtin_data.h file, first compile the mkbuiltin.c program, then run:
<blockquote><pre>
mkbuiltin.exe diff.tcl <i>OtherFiles...</i> >builtin_data.h
</pre></blockquote>
At the time of this writing, the "diff.tcl" script (a Tcl/Tk script used
to generate implement --tk option on the diff command) is the only resource
file processed using mkbuiltin.exe. However, new resources will likely be
added using this facility in future versions of Fossil.
<a name="preprocessing"></a>
<h1>4.0 Preprocessing</h1>
There are three preprocessors for the Fossil sources. The mkindex
and translate preprocessors can be run in any order. The makeheaders
preprocessor must be run after translate.
<h2>4.1 The mkindex preprocessor</h2>
The mkindex program scans the "src.c" source files looking for special
comments that identify routines that implement various Fossil commands,
web interface methods, and help text comments. The mkindex program
generates some C code that Fossil uses in order to dispatch commands and
HTTP requests and to show on-line help. Compile the mkindex program
from the mkindex.c source file. Then run:
<blockquote><pre>
./mkindex src.c >page_index.h
</pre></blockquote>
Note that "src.c" in the above is a stand-in for the (79) regular source
files of Fossil - all source files except for the exceptions described in
section 2.0 above.
The output of the mkindex program is a header file that is #include-ed by
the main.c source file during the final compilation step.
<h2>4.2 The translate preprocessor</h2>
The translate preprocessor looks for lines of source code that begin
with "@" and converts those lines into string constants or (depending on
context) into special "printf" operations for generating the output of
an HTTP request. The translate preprocessor is a simple C program whose
sources are in the translate.c source file. The translate preprocess
is run on each of the other ordinary source files separately, like this:
<blockquote><pre>
./translate src.c >src_.c
</pre></blockquote>
In this case, the "src.c" file represents any single source file from the
set of ordinary source files as described in section 2.0 above. Note that
each source file is translated separately. By convention, the names of
the translated source files are the names of the input sources with a
single "_" character at the end. But a new makefile can use any naming
convention it wants - the "_" is not critical to the build process.
After being translated, the output files (the "src_.c" files) should be
used for all subsequent preprocessing and compilation steps.
<h2>4.3 The makeheaders preprocessor</h2>
For each C source module "src.c", there is an automatically generated
header module "src.h" that contains all of the datatype and procedure
declarations needed by the source module. These header files are generated
automatically by the makeheaders program. The sources to makeheaders
are contained in a single file "makeheaders.c". Additional documentation
on makeheaders can be found in [../src/makeheaders.html | src/makeheaders.html].
The makeheaders program is run once. It scans all inputs source files and
generates header files for each one. Note that the sqlite3.c and shell.c
source files are not scanned by makeheaders. Makeheaders only runs over
"ordinary" source files, not the exceptional source files. However,
makeheaders also uses some extra header files as input. The general format
is like this:
<blockquote><pre>
makeheaders src_.c:src.h sqlite3.h th.h VERSION.h
</pre></blockquote>
In the example above the "src_.c" and "src.h" names represent all of the
(79) ordinary C source files, each as a separate argument.
<h1>5.0 Compilation</h1>
After all generated files have been created and all ordinary source files
have been preprocessed, the generated and preprocessed files can be
combined into a single executable using a C compiler. This can be done
all at once, or each preprocessed source file can be compiled into a
separate object code file and the resulting object code files linked
together in a final step.
Some files require special C-preprocessor macro definitions.
When compiling sqlite.c, the following macros are recommended:
* -DSQLITE_OMIT_LOAD_EXTENSION=1
* -DSQLITE_ENABLE_DBSTAT_VTAB=1
* -DSQLITE_ENABLE_FTS4=1
* -DSQLITE_ENABLE_LOCKING_STYLE=0
* -DSQLITE_THREADSAFE=0
* -DSQLITE_DEFAULT_FILE_FORMAT=4
* -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
The first three symbol definitions above are required; the others are merely
recommended. Extension loading is omitted as a security measure. The dbstat
virtual table is needed for the [/help?cmd=/repo-tabsize|/repo-tabsize] page.
FTS4 is needed for the search feature. Fossil is single-threaded so mutexing
is disabled in SQLite as a performance enhancement. The
SQLITE_ENABLE_EXPLAIN_COMMENTS option makes the output of "EXPLAIN" queries
in the "[/help?cmd=sqlite3|fossil sql]" command much more readable.
When compiling the shell.c source file, these macros are required:
* -Dmain=sqlite3_main
* -DSQLITE_OMIT_LOAD_EXTENSION=1
The "main()" routine in the shell must be changed into sqlite3_main()
to prevent it from colliding with the real main() in Fossil, and to give
Fossil an entry point to jump to when the
[/help/sqlite3 | fossil sql] command is invoked.
All the other source code files can be compiled without any special
options.
<h1>6.0 Linkage</h1>
Fossil needs to be linked against [http://www.zlib.net | zlib]. If
the HTTPS option is enabled, then it will also need to link against
the appropriate SSL implementation. And, of course, Fossil needs to
link against the standard C library. No other libraries or external
dependences are used.
<h1>7.0 See Also</h1>
* [./tech_overview.wiki | A Technical Overview Of Fossil]
* [./adding_code.wiki | How To Add Features To Fossil]