Fossil

Check-in [c429b524]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add documentation on how the build process works.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:c429b524654c59393e67f6d5965dbf714e4d5657
User & Date: drh 2010-12-30 14:03:14
Context
2010-12-30
14:41
Add a -local option to the "date" TH1 command. This allows a custom ticket-edit script to use localtime instead of UTC for the date stamp for when a comment is added. Also modify the default ticket-edit script to show UTC after the date stamp, to avoid ambiguity. Ticket [5c1490e26b8b2f8]. check-in: e0762527 user: drh tags: trunk
14:03
Add documentation on how the build process works. check-in: c429b524 user: drh tags: trunk
02:15
make correction on image/png magic for mimetype detection check-in: ff53d5ab user: bch tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Added www/makefile.wiki.













































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
<title>The Fossil Build Process</title>

<h1>1.0 Introduction</h1>

The build process for Fossil is tricky in that the source code
needs to be processed by three different preprocessor programs
before it is compiled.  Most users will download a precompiled
binary so this is of no consequence to them, and even those who
want to compile the code themselves can use one of the existing
makefiles.  So must people do not need to be concerned with the
build complexities of Fossil.  But hard-cord developers who desire
a deep understanding of how Fossil is put together can benefit
from reviewing this article.

<h1>2.0 Source Code Tour</h1>

The source code for Fossil is found in the src/ subdirectory of the
source tree.  The src/ subdirectory contains all code, including
the code for the separate preprocessor programs.

Each preprocessor program is a separate C program implemented in
a single file of C source code.  The three preprocessor programs
are:

  1.  mkindex.c
  2.  translate.c
  3.  makeheaders.c

Fossil makes use of [http://www.sqlite.org/ | SQLite] for on-disk
storage.  The SQLite implementation is contained in three source
code files that do not participate in the preprocessing steps.
These three files that implement SQLite are:

  4.  sqlite3.c
  5.  sqlite3.h
  6.  shell.c

The sqlite3.c and sqlite3.h source files are byte-for-byte copies of a
standard [http://www.sqlite.org/amalgamation.html | amalgamation].
The shell.c source file is code for the SQLite
[http://www.sqlite.org/sqlite.html | command-line shell] that is used
to help implement the [/help/sqlite3 | fossil sql] command.  The
shell.c source file has been modified slightly from the standard
shell.c file in the SQLite release.  Search for "Fossil Patch" in
the shell.c source file of Fossil to see the changes.

The TH1 script engine is implemented using files:

  7.  th.c
  8.  th.h

These two files are imports like the SQLite source files, 
and so are not preprocessed.

The src/ subdirectory also contains documentation about the
makeheaders preprocessor program:

  9.  [../src/makeheaders.html | makeheaders.html]

Click on the link to read this documentation.  In addition there is
a [http://www.tcl.tk/ | Tcl] script used to build the various makefiles:

  10.  makemake.tcl

Running this Tcl script will automatically regenerate all makefiles.
In order to add a new source file to the Fossil implementation, simply
edit makemake.tcl to add the new filename, then rerun the script, and
all of the makefiles for all targets will be rebuild.

Finally, there is one of the makefiles generated by makemake.tcl:

  11.  main.mk

The main.mk makefile is invoked from the Makefile in the top-level
directory.  The main.mk is generated by makemake.tcl and should not
be hand edited.  Other makefiles generated by makemake.tcl are in 
other subdirectories (currently all in the win/ subdirectory).

All the other files in the src/ subdirectory (79 files at the time of
this writing) are C source code files that are subject to the
preprocessing steps described below.  In the sequel, we will call these
other files "src.c" in order to have a convenient name.   The reader
should understand that whenever "src.c" or "src.h" is used in the text
that follows, we really mean all (79) other source files other than
the exceptions described above.

<h1>3.0 Automatically generated files</h1>

The "VERSION.h" header file contains some C preprocessor macros that
identify the version of Fossil that is to be build.  The VERSION.h file
generated automatically from information extracted from the "manifest"
and "manifest.uuid" source files in the root directory of the source tree.
(The "manifest" and "manifest.uuid" files are automatically generated and
updated by Fossil itself.  See the [/help/setting | fossil set manifest]
command for additional information.)

Under unix, there is an AWK script that converts manifest and manifest.uuid
into VERSION.h.  For windows, there is a C program: win/version.c.  
(Unix builds could also used the version.c convert, if desired, but AWK
seems easier there.)  To run the VERSION.h generator, first compile
the win/version.c source file into a command-line program (named "version.exe")
than run:

<blockquote><pre>
version.exe manifest.uuid manifest VERSION.h
</pre></blockquote>

The pathnames in the above command might need to be adjusted to get the
directories right.  The point is that the manifest.uuid and manifest files
in the root of the source tree are the first two arguments and the name of
the generated VERSION.h file is the third and final argument.

<h1>4.0 Preprocessing</h1>

There are three preprocessors for the Fossil sources.  The mkindex
and translate preprocessors can be run in any order.  The makeheaders
preprocessor has to be run after translate.

<h2>4.1 The mkindex preprocessor</h2>

The mkindex program scans the "src.c" source files looking for special
comments that identify routines that implement of various Fossil commands,
web interface methods, and help text comments.  The mkindex program
generates some C code that Fossil uses in order to dispatch commands and
HTTP requests and to show on-line help.  Compile the mkindex program
from the mkindex.c source file.  Then run:

<blockquote><pre>
./mkindex src.c >page_index.h
</pre></blockquote>

Note that "src.c" in the above is a stand-in for the (79) regular source
files of Fossil - all source files except for the exceptions described in
section 2.0 above.

The output of the mkindex program is a header file that is #include-ed by
the main.c source file during the final compilation step.  

<h2>4.2 The translate preprocessor</h2>

The translate preprocessor looks for lines of source code that begin
with "@" and converts those lines into string constants or (depending on
context) into special "printf" operations for generating the output of
an HTTP request.  The translate preprocessor is a simple C program whose
sources are in the translate.c source file.  The translate preprocess
is run on each of the other ordinary source files separately, like this:

<blockquote><pre>
./translate src.c >src_.c
</pre></blockquote>

In this case, the "src.c" file represents any single source file from the
set of ordinary source files as described in section 2.0 above.  Note that
each source file is translated separately.  By convention, the names of
the translated source files are the names of the input sources with a 
single "_" character at the end.  But a new makefile can use any naming
convention it wants - the "_" is not critical to the build process.

After being translated, the output files (the "src_.c" files) should be
used for all subsequent preprocessing and compilation steps.

<h2>4.3 The makeheaders preprocessor</h2>

For each C source module "src.c", there is an automatically generated
header module "src.h" that contains all of the datatype and procedure
declarations needed by the source module.  These header files are generated
automatically by the makeheaders program.  The sources to makeheaders
are contained in a single file "makeheaders.c".  Additional documentation
on makeheaders can be found in [../src/makeheaders.html | src/makeheaders.html].

The makeheaders program is run once.  It scans all inputs source files and
generates header files for each one.  Note that the sqlite3.c and shell.c
source files are not scanned by makeheaders.  Makeheaders only runs over
"ordinary" source files, not the exceptional source files.  However,
makeheaders also uses some extra header files as input.  The general format
is like this:

<blockquote><pre>
makeheaders src_.c:src.h sqlite3.h th.h VERSION.h
</pre></blockquote>

In the example above the "src_.c" and "src.h" names represent all of the
(79) ordinary C source files, each as a separate argument.  

<h1>5.0 Compilation</h1>

After all generated files have been created and all ordinary source files
have been preprocessed, the generated and preprocessed files can be 
combined into a single executable using a C compiler.  This can be done
all at once, or each preprocessed source file can be compiled into a
separate object code file and the resulting object code files linked
together in a final step.

Some files require special C-preprocessor macro definitions.
When compiling sqlite.c, the following macros are recommended:

  *  -Dlocaltime=fossil_localtime
  *  -DSQLITE_OMIT_LOAD_EXTENSION=1
  *  -DSQLITE_ENABLE_LOCKING_STYLE=0
  *  -DSQLITE_THREADSAFE=0
  *  -DSQLITE_DEFAULT_FILE_FORMAT=4

The first and second symbol definitions above are required; the others
are merely recommended.  The "localtime()" library function in SQLite must
be redefined to invoke fossil_localtime() instead.  The fossil_localtime()
routine will invoke either gmtime() or localtime() depending on the 
"Use UTC" setting for the fossil repository.  Extension loading is omitted
as a security measure.  Fossil is single-threaded so mutexing is disabled
in SQLite as a performance enhancement.

When compiling the shell.c source file, these macros are required:

  *  -Dmain=sqlite3_main
  *  -DSQLITE_OMIT_LOAD_EXTENSION=1

The "main()" routine in the shell must be changed into sqlite3_main()
to prevent it from colliding with the real main() in Fossil, and to give
Fossil an entry point to jump to when the 
[/help/sqlite3 | fossil sql] command is invoked.

All the other source code files can be compiled without any special
options.

<h1>6.0 Linkage</h1>

Fossil needs to be linked against [http://www.zlib.net | zlib].  If
the HTTPS option is enabled, then it will also need to link against
the appropriate SSL implementation.  And, of course, Fossil needs to
link against the standard C library.  No other libraries or external
dependences are used.

Changes to www/mkindex.tcl.

18
19
20
21
22
23
24

25
26
27
28
29
30
31
  embeddeddoc.wiki {Embedded Project Documentation}
  event.wiki {Events}
  faq.wiki {Frequently Asked Questions}
  fileformat.wiki {Fossil File Format}
  fossil-v-git.wiki {Fossil Versus Git}
  index.wiki {Home Page}
  inout.wiki {Import And Export To And From Git}

  password.wiki {Password Management And Authentication}
  pop.wiki {Principles Of Operations}
  qandc.wiki {Questions And Criticisms}
  quickstart.wiki {Fossil Quick Start Guide}
  quotes.wiki
      {Quotes: What People Are Saying About Fossil, Git, and DVCSes in General}
  selfcheck.wiki {Fossil Repository Integrity Self Checks}







>







18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
  embeddeddoc.wiki {Embedded Project Documentation}
  event.wiki {Events}
  faq.wiki {Frequently Asked Questions}
  fileformat.wiki {Fossil File Format}
  fossil-v-git.wiki {Fossil Versus Git}
  index.wiki {Home Page}
  inout.wiki {Import And Export To And From Git}
  makefile.wiki {The Fossil Build Process}
  password.wiki {Password Management And Authentication}
  pop.wiki {Principles Of Operations}
  qandc.wiki {Questions And Criticisms}
  quickstart.wiki {Fossil Quick Start Guide}
  quotes.wiki
      {Quotes: What People Are Saying About Fossil, Git, and DVCSes in General}
  selfcheck.wiki {Fossil Repository Integrity Self Checks}

Changes to www/permutedindex.wiki.

3
4
5
6
7
8
9

10
11
12
13
14
15
16
..
59
60
61
62
63
64
65

66
67
68
69
70
71
72
..
76
77
78
79
80
81
82

83
84
85
86
87
88
89
90
91
92
93
<ul>
<li><a href="event.wiki">Events</a></li>
<li><a href="delta_encoder_algorithm.wiki">Algorithm &#151; Fossil Delta Encoding</a></li>
<li><a href="faq.wiki">Asked Questions &#151; Frequently</a></li>
<li><a href="password.wiki">Authentication &#151; Password Management And</a></li>
<li><a href="branching.wiki">Branching, Forking, Merging, and Tagging</a></li>
<li><a href="bugtheory.wiki">Bug Tracking In Fossil</a></li>

<li><a href="build.wiki">Building and Installing Fossil</a></li>
<li><a href="checkin_names.wiki">Checkin And Version Names</a></li>
<li><a href="selfcheck.wiki">Checks &#151; Fossil Repository Integrity Self</a></li>
<li><a href="concepts.wiki">Concepts &#151; Fossil Core</a></li>
<li><a href="server.wiki">Configure A Fossil Server &#151; How To</a></li>
<li><a href="shunning.wiki">Content From Fossil &#151; Deleting</a></li>
<li><a href="concepts.wiki">Core Concepts &#151; Fossil</a></li>
................................................................................
<li><a href="checkin_names.wiki">Names &#151; Checkin And Version</a></li>
<li><a href="pop.wiki">Operations &#151; Principles Of</a></li>
<li><a href="index.wiki">Page &#151; Home</a></li>
<li><a href="password.wiki">Password Management And Authentication</a></li>
<li><a href="quotes.wiki">People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What</a></li>
<li><a href="stats.wiki">Performance Statistics</a></li>
<li><a href="pop.wiki">Principles Of Operations</a></li>

<li><a href="embeddeddoc.wiki">Project Documentation &#151; Embedded</a></li>
<li><a href="sync.wiki">Protocol &#151; The Fossil Sync</a></li>
<li><a href="faq.wiki">Questions &#151; Frequently Asked</a></li>
<li><a href="qandc.wiki">Questions And Criticisms</a></li>
<li><a href="quickstart.wiki">Quick Start Guide &#151; Fossil</a></li>
<li><a href="quotes.wiki">Quotes: What People Are Saying About Fossil, Git, and DVCSes in General</a></li>
<li><a href="selfhost.wiki">Repositories &#151; Fossil Self Hosting</a></li>
................................................................................
<li><a href="selfhost.wiki">Self Hosting Repositories &#151; Fossil</a></li>
<li><a href="server.wiki">Server &#151; How To Configure A Fossil</a></li>
<li><a href="quickstart.wiki">Start Guide &#151; Fossil Quick</a></li>
<li><a href="stats.wiki">Statistics &#151; Performance</a></li>
<li><a href="sync.wiki">Sync Protocol &#151; The Fossil</a></li>
<li><a href="custom_ticket.wiki">System &#151; Customizing The Ticket</a></li>
<li><a href="branching.wiki">Tagging &#151; Branching, Forking, Merging, and</a></li>

<li><a href="sync.wiki">The Fossil Sync Protocol</a></li>
<li><a href="webui.wiki">The Fossil Web Interface</a></li>
<li><a href="theory1.wiki">Thoughts On The Design Of The Fossil DVCS</a></li>
<li><a href="custom_ticket.wiki">Ticket System &#151; Customizing The</a></li>
<li><a href="bugtheory.wiki">Tracking In Fossil &#151; Bug</a></li>
<li><a href="checkin_names.wiki">Version Names &#151; Checkin And</a></li>
<li><a href="fossil-v-git.wiki">Versus Git &#151; Fossil</a></li>
<li><a href="webui.wiki">Web Interface &#151; The Fossil</a></li>
<li><a href="quotes.wiki">What People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes:</a></li>
<li><a href="wikitheory.wiki">Wiki In Fossil</a></li>
</ul>







>







 







>







 







>











3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
..
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
..
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
<ul>
<li><a href="event.wiki">Events</a></li>
<li><a href="delta_encoder_algorithm.wiki">Algorithm &#151; Fossil Delta Encoding</a></li>
<li><a href="faq.wiki">Asked Questions &#151; Frequently</a></li>
<li><a href="password.wiki">Authentication &#151; Password Management And</a></li>
<li><a href="branching.wiki">Branching, Forking, Merging, and Tagging</a></li>
<li><a href="bugtheory.wiki">Bug Tracking In Fossil</a></li>
<li><a href="makefile.wiki">Build Process &#151; The Fossil</a></li>
<li><a href="build.wiki">Building and Installing Fossil</a></li>
<li><a href="checkin_names.wiki">Checkin And Version Names</a></li>
<li><a href="selfcheck.wiki">Checks &#151; Fossil Repository Integrity Self</a></li>
<li><a href="concepts.wiki">Concepts &#151; Fossil Core</a></li>
<li><a href="server.wiki">Configure A Fossil Server &#151; How To</a></li>
<li><a href="shunning.wiki">Content From Fossil &#151; Deleting</a></li>
<li><a href="concepts.wiki">Core Concepts &#151; Fossil</a></li>
................................................................................
<li><a href="checkin_names.wiki">Names &#151; Checkin And Version</a></li>
<li><a href="pop.wiki">Operations &#151; Principles Of</a></li>
<li><a href="index.wiki">Page &#151; Home</a></li>
<li><a href="password.wiki">Password Management And Authentication</a></li>
<li><a href="quotes.wiki">People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What</a></li>
<li><a href="stats.wiki">Performance Statistics</a></li>
<li><a href="pop.wiki">Principles Of Operations</a></li>
<li><a href="makefile.wiki">Process &#151; The Fossil Build</a></li>
<li><a href="embeddeddoc.wiki">Project Documentation &#151; Embedded</a></li>
<li><a href="sync.wiki">Protocol &#151; The Fossil Sync</a></li>
<li><a href="faq.wiki">Questions &#151; Frequently Asked</a></li>
<li><a href="qandc.wiki">Questions And Criticisms</a></li>
<li><a href="quickstart.wiki">Quick Start Guide &#151; Fossil</a></li>
<li><a href="quotes.wiki">Quotes: What People Are Saying About Fossil, Git, and DVCSes in General</a></li>
<li><a href="selfhost.wiki">Repositories &#151; Fossil Self Hosting</a></li>
................................................................................
<li><a href="selfhost.wiki">Self Hosting Repositories &#151; Fossil</a></li>
<li><a href="server.wiki">Server &#151; How To Configure A Fossil</a></li>
<li><a href="quickstart.wiki">Start Guide &#151; Fossil Quick</a></li>
<li><a href="stats.wiki">Statistics &#151; Performance</a></li>
<li><a href="sync.wiki">Sync Protocol &#151; The Fossil</a></li>
<li><a href="custom_ticket.wiki">System &#151; Customizing The Ticket</a></li>
<li><a href="branching.wiki">Tagging &#151; Branching, Forking, Merging, and</a></li>
<li><a href="makefile.wiki">The Fossil Build Process</a></li>
<li><a href="sync.wiki">The Fossil Sync Protocol</a></li>
<li><a href="webui.wiki">The Fossil Web Interface</a></li>
<li><a href="theory1.wiki">Thoughts On The Design Of The Fossil DVCS</a></li>
<li><a href="custom_ticket.wiki">Ticket System &#151; Customizing The</a></li>
<li><a href="bugtheory.wiki">Tracking In Fossil &#151; Bug</a></li>
<li><a href="checkin_names.wiki">Version Names &#151; Checkin And</a></li>
<li><a href="fossil-v-git.wiki">Versus Git &#151; Fossil</a></li>
<li><a href="webui.wiki">Web Interface &#151; The Fossil</a></li>
<li><a href="quotes.wiki">What People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes:</a></li>
<li><a href="wikitheory.wiki">Wiki In Fossil</a></li>
</ul>